🏥 Healthcare, life science, and COVID
As I’ve discussed in several previous editions of this newsletter, the pharma industry is broadly under-indexed on AI-first computational and robotic platform approaches to drug discovery and development (DDD). Several startups in this field are making strong progress, e.g. Exscientia, LabGenius, Cellarity, Recursion, Insitro, and others. A piece
in Nature Biotechnology
surveyed this landscape and focused on how iterative cycles of machine learning, empirical wet-lab experimentation, and human feedback can accelerate DDD. Pharma is of course evolving too. Of note, Roche appointed Aviv Regev
of the Broad Institute of Harvard and MIT as the new Head of Genentech Research and Early Development. This is noteworthy because Aviv has made a storied career in data-driven biology, including single-cell RNA sequencing and ML-based analytical pipelines, and while also co-chairing the Human Cell Atlas project.
The COVID pandemic has shown that technology startups collaborating with clinical research centers play an important role in public health. In a citizen science study
with >2.6M participants reported in Nature Medicine
, the health science company ZOE, Mass General Hospital, and KCL’s NHS Trust were able to validate “non-canonical” COVID symptoms far earlier than governmental health services. In particular, a combination of a) loss of smell+taste, b) fatigue, c) a persistent cough and d) a loss of appetite is most strongly correlated with COVID-19, adjusting for age, sex and BMI. Using a model that was trained to predict COVID from reported symptoms of 18,401 participants that had undergone a COVID RT-PCR test, the study predicted that close to 18% of symptom reporting participants are likely to have COVID.
Meanwhile, multiple research groups and companies have run and released drug screening data
to find potential COVID treatments. However, from the 200+ active compounds that several studies reported on, only 32 were identified as being active in more than one study. Many of the drugs used in COVID clinical studies do not have published in vitro
or in vivo
studies in bioRxiv, which suggests that we know little about how they work. Without screening standards, it is hard to draw comparisons across studies too.
NinesAI, a Palo Alto-based company, gained 510(k) FDA clearance
for their automated radiological review that analyses CT images of the head region to detect brain bleeds. Detecting this condition quickly is imperative to avoid long-term damage and death. Approximately half of the 30-day mortality occurs within the first 24 hours.
🌎 The (geo)politics of AI
The US government signed an extension
of last year’s executive order that barred US companies from using telecoms equipment made by firms that present a national security risk (such as Huawei and ZTE Corp). The government’s Department of Commerce also added 24 Chinese companies and institutions
to a sanction list for “supporting the procurement of items for military end-use in China”. A further 8 companies and the Institute for Forensic Science were placed on a second list
that restricts access to US technology because they are “complicit in human rights violations and abuses…against Uygurs, ethnic Kazakhs, and other members of Muslim minority groups in the Xinjiang Uygur Autonomous Region”. The list includes Qihoo 360 (antivirus software and web browser), Cloudminds (RPA software), and CloudWalk (facial recognition software). Even so, CloudWalk raised $254M from Chinese provincial and municipal funds as it eyes a public listing on the Shanghai exchange this year. The company was incubated at the Chongqing Research Institute and was deeply involved in guiding the national strategy for facial recognition. SenseTime, another leading Chinese facial recognition startup is in the market to raise $1B.
In January the UK government decided to cap Huawei’s 5G equipment footprint to 35% and barred its use in the critical core of mobile networks where data is stored and routed. Now, the government is drawing up a 3-year plan to remove
Huawei from 5G networks entirely.
TSMC, the world’s largest semiconductor fabricator, said it would spend $12B
to create a chip fab in Arizona. This is undoubtedly a geopolitically-influenced decision in the backdrop of US-China trade tensions and Trump’s aforementioned executive order. The factory would focus on TSMC’s 5-nanometer process.
There’s been a flurry of government and defense contractor agreements around AI technology. This includes 1) a 5-year, $800M contract
between the US DoD’s Joint AI Center and Booze Allen Hamilton, 2) the DoD’s Defense Innovation Unit selecting
Google Cloud to build a solution to detect, protect against, and respond to cyber threats, and 3) Canada’s DarwinAI announcing a strategic collaboration
with aerospace contractor Lockheed Martin around explainable AI solutions.
🚗 Autonomous everything
The latest in the self-driving world is a mix of layoffs, consolidation, fundraising, and talent poaching due to the COVID-induced industry timeout
. GM’s Cruise laid off 8%
of its staff (circa 150 employees) as it sought to cut costs through COVID. The company added Regina Dugan
(Google’s former head of Advanced Technology and Products Group and DARPA Director) to its board. GM also announced its “Super Cruise
” system, which is a hands-free driving feature on pre-mapped highways that takes aim at Tesla’s AutoPilot.
Reports surfaced that Zoox, the startup that reimagined the vehicle for the era of autonomy, is in trouble. Word on the street is that Amazon is soon set to acquire the business for around $1B, which would be the sum of the venture capital that Zoox raised. While the deal is in the process of closing, competitors such as Cruise are poaching
As others cut their headcount, Aurora reported passing the 500 employee mark. The company added two senior engineering leaders both of whom developed their academic and professional careers in Europe (Free University of Berlin and KTH in Stockholm). Interestingly, Aurora is launching an in-house university
to upskill their employees for roles that are supply-constrained on the market.
Meanwhile, the latest poll
of 1,200 Americans by a coalition of industry payers and non-profits called Partners for Automated Vehicle Education revealed that ¾ Americans say that AV technology “is not ready for primetime”. Of note, 48% said they would not get in a self-driving taxi. Having said that, half of the people polled said they owned vehicles with ADAS features and responded favorably to having a vehicle that supports their driving as long as the driver is in full control.
its integration of Luminar’s LiDAR system that will go into production from 2022. Luminar’s Iris system offers up to 500m range, which is set to help Volvo deliver ADAS on dedicated highway stretches.
For a roundup of the development stage and progress of major self-driving players, check out this piece
💪 The giants
Automation in the enterprise is a very hot topic. Vendors across the stack from startups to public companies are selling enterprises on the virtues of implementing software to automate repetitive workflows. This software often comes in the form of robotic process automation (RPA), low- or even no-code builders, and AI-based features such as document digitization (OCR). A recent survey
of 796 executives by Bain found that while companies expect to double their use of automation technologies in the next two years, 44% of respondents said their automation projects have not achieved the savings they expected. There are a few reasons. Although automation can cut labor time by 20-30%, the median payback for these savings is 13 to 18 months. Those organizations that see higher levels of savings from automation tend to 1) have C-level sponsorship for the project (i.e. establishing automation as a key priority), 2) have a “center of excellence” for automation (i.e. centralized coordination and capabilities), and 3) spend >20% of their IT budget on automation.
Twitter announced the appointment
of Fei-Fei Li to its Board of Directors. Fei-Fei is known for her work on computer vision (e.g. ImageNet), as a Professor at Stanford, and most recently as Head of AI for Google Cloud.
Google’s open-source project, TensorFlow, has surpassed
100 million downloads since its launch in 2015. In the last month alone, there were 10 million downloads. Nonetheless, it appears that Facebook’s PyTorch has overtaken
TensorFlow in research code.
Facebook is also upping the ante
on their marketplace product thanks to new computer vision models from Facebook AI and the Grokstyle team that was acquired in 2019. They deployed the GrokNet
computer vision system that can identify fine-grained product attributes across billions of photos to help marketplace tag photos and make photos shoppable on Facebook Pages. The release also introduced Rotating View, which lets smartphone users capture multi-dimensional panoramic views of their listings. Together, the company hopes to drive more transactions on its marketplace.
In response to a new Greenpeace report
that detailed 14 separate contracts between Amazon, Microsoft, and Google with major oil firms, Google responded
by saying they “will not…build customer AI algorithms to facilitate upstream extraction in the oil and gas industry”. The company’s Cloud revenues from oil and gas customers were roughly $65M in 2019 (from a pool worth $113B according to HG Insights). Greenpeace isn’t satisfied because they don’t see how upstream extraction is a good use case for AI.
At the Microsoft Build 2020 conference, OpenAI demonstrated a pretty impressive code
generation demo of a large language model trained on thousands of GitHub repositories. The developer writes prompts that instruct what a program should do, and the language model outputs the correct code. OpenAI also released its third generation GPT
language model that has 175 billion parameters (10x more than any previous non-sparse language model).
the rollout of two new AR features that create more realistic, real-world experiences. This includes improved occlusion detection and the use of Portal Scanning videos to generate dynamic 3D maps of physical places. This helps the company capture crowdsourced data of the real world from which to build semantic and depth maps to empower AR features. We can expect a ton more to come from Niantic as it continues to deepen its talent bench in machine learning and AR.
It’s no secret in the industry that the compute requirements for training modern AI systems are going up and to the right. A recent analysis
by OpenAI showed there are two distinct eras in the timeline of AI training compute requirements. The first is pre-2012
, i.e. pre/early-deep learning (Belief Networks, BiLSTMs, RNNs), and the second is post-2013
, i.e. the age of modern deep learning (ResNet, BERT, AlphaGoZero). At the same time, there appears to be a concomitant reduction in algorithmic efficiency
. That is to say, modern deep learning systems can achieve AlexNet performance with 44x less compute.
NVIDIA has unveiled
its Ampere-based DGX A100 chip that contains 54 billion transistors and can deliver 5 petaflops of performance (20x the Volta chip) from its 8 A100 Tensor Core GPUs and 320Gb of memory. The chips are made with TSMC’s 7-nanometer process and cost $199,000. NVIDIA also released a whitepaper on their A100 Tensor Core GPU architecture here
, which they call “the greatest generational leap in NVIDIA GPU accelerated computing ever”. Coupling the depth of NVIDIA’s software stack with these new chips makes NVIDIA a serious contender to reckon with.
an integrated AI processor and image sensor product that is positioned to run ML workloads directly on their cameras. This could be interesting in photography!
About a year ago, Microsoft announced a $1B strategic investment into OpenAI to develop an Azure-based supercomputer for AI workloads. Now, Microsoft has announced
the result: a 285,000-core supercomputer running on Azure that is exclusively available to OpenAI.