🦔Your guide to AI: April 2021

Hi everyone,

Welcome to Your guide to AI. Here you’ll find an analytical (and opinionated) narrative covering key developments in AI over April 2021. 

If you’d like to chat about something you’re working on, share some news before it happens, or have feedback on the issue, just hit reply! I’ll be sending through the audio edition next weekend. 

If you enjoyed the read, I’d appreciate you hitting forward to a couple of friends 🙏

Last time we discussed how computational techniques in drug discovery often focus on very early steps of the value chain and that there are few examples of drugs that made it into clinical testing. Good timing! Exscientia, an Oxford-based AI-first pharma company, announced that their second AI-designed drug has entered Phase 1 testing. This drug, discovered in collaboration with Evotec (an early investor in the company) and further validated in collaboration with Allcyte (an Air Street portfolio company), is also the first AI-designed immuno-oncology drug to enter human trials. It targets the A2a receptor, which is responsible for driving advanced solid tumors by suppressing the anti-tumor immune cell activity of T-cells. 

Turning to over clinical medicine, one could say that each practitioner is his/her own AI model. We’re trained on the cases we study in school and those that we encounter in the field. Wouldn’t it make more sense to have model expertise disseminate much quicker and at almost zero marginal cost to level quality of care from advanced systems to emerging ones? In a way, this is what federated learning could unlock - we could send naive or locally-trained models to distant sites such that they can learn from new cases (data distributions) and return more robust. Before we get there, “simply” distributing trained AI systems to lesser developed medical systems can do the trick. That’s what Qure.ai did in a collaboration with the 130-bed Baptist Christian Hospital at the Himalayan foothills in Assam, India. Their system drove automated CT brain interpretation to rapidly and easily figure out the next best treatment options for stroke patients and to examine the implications for clinical outcomes. 

Even so, the AI for the medical imaging sector is still in its infancy. A study of 100 commercially available products and their scientific evidence found that 64% lacked peer-reviewed evidence on efficacy while only 18% had demonstrated (potential) clinical impact. 

Over in the animal kingdom, the New Yorker ran a story about various data-driven efforts to build animal-to-human communication systems. Some suggest that using machine learning to model species-specific communication in an unsupervised manner, while others are working on machine translation in a more structured communication environment, i.e. seeing how animals communicate in response to human utterances. 

Following on from last month’s discussion of the US National Security Commission report, this month is all about the European Commission’s draft Regulation on AI. The goal is to harmonize legal frameworks across the EU to facilitate innovation, investment, and safeguarding of fundamental rights and AI safety. The report calls out prohibited AI practices such as large-scale surveillance and adverse behavioral influencing, high-risk AI systems such as credit scoring and medical devices that must be used with specific controls in place, and lower-risk AI systems that are subject to a transparency regime. Hard to see how clearly one can separate one application from another into these risk buckets. Their definitions of AI systems are very broad-ranging from rules, statistics, and deep learning with some joking that the “EU is proposing to regulate the use of Bayesian estimation”. The draft also introduces requirements such as risk management, testing, and real-world monitoring, human review security, accountability, and CE marking. It calls for the establishment of a European AI Board that should advise and assist the commission on matters of AI regulation and leaves member state authorities to conduct market surveillance and control these AI systems. In a hat tip to GDPR, the report defines monetary sanctions for infringement up to €10m - €30m or 2-6% of global annual turnover. There are of course huge ramifications of this legislation and many years of heavy lobbying ahead before a form of this plan is put into practice. 

Over in the US, the Commerce Department grew its list of blacklisted Chinese entities. It  adding seven firms that are allegedly “involved with building supercomputers used by China’s military actors, its destabilizing military modernization efforts, and/or weapons of mass destruction programs.“ This means they’ll have to apply for licenses to receive items from US-based suppliers. 

Meanwhile, Biden’s first proposed budget announcement called for a funding boost of almost 20% across the board for the CDC, NSF, NIH, DOE, and other departments responsible for the country’s public health system and biomedical research. This was a welcome policy stance after many years of the Trump administration calling to slash science funding.

Back in the UK, the Secretary of State for Digital, Culture, Media, and Sport flexed their powers under the Enterprise Act 2002 by issuing a “public interest intervention notice” to intervene in the sale of Arm to NVIDIA on “national security grounds”. The Competition and Markets Authority is now due to preparing a report on the competition and natural security aspects of this transaction by the end of July 2021. 

Sailing - much like Formula 1 and other high-performance racing sports - relies on complex optimization of vehicle design to deliver the fastest times on race day. Emirates Team New Zealand discussed their use of reinforcement learning within an in-house sailing simulator to evaluate thousands of hydrofoil design concepts (instead of hundreds). The piece, written by McKinsey’s Quantum Black, explores other RL applications in pharma, retail, telecoms, and oil and gas. We can expect RL techniques to shine as we learn more about how to describe complex optimization tasks and develop high-fidelity simulators. 

NVIDIA held its annual GTC during which it announced the company’s first data center CPU, Grace, that is built on Arm technology and targeted at HPC and AI applications such as large language models. It is expected to become available in 2023 as a means of complementing their GPU offerings as a co-processor. NVIDIA also announced a partnership with AWS and Ampere Computing to expand Arm into the cloud, along with Omniverse, a cloud-native GPU-based simulation platform for 3D virtual worlds and science. For more announcements, check out this summary post. 

While the shortage of semiconductors continues, the sales of wafer processing equipment that is required to manufacture chips have surged to an all-time record of $71B in 2020, up 19% from the year before (along with the companies’ respective share prices). The driver is China’s domestic fab industry, the massive investments by TSMC and Samsung Semiconductor, and the rising costs of lithography equipment. 

Cerebras released their second-generation wafer-scale engine, which comes in with 2.6 trillion transistors on TSMC’s 7nm process. Compared to its first-gen system, everything is 2x greater: cores, transistors, density, onboard SRAM, memory, and fabric bandwidth. 

Since disclosing its focus on trucking versus passenger transport, Aurora announced its collaboration with Volvo trucks. The Swedish automotive giant formed an Autonomous Solutions business to develop Level-4 trucks and will make use of the Aurora Driver and the company’s 300m LiDAR solution.

Meanwhile, in the UK, the government said that it has “set out” how to legally define automated lane-keeping systems that could go live on public roads by the end of 2021. 

Cloudflare has built up a huge edge network in over 200 cities worldwide and four years ago launched Workers, which lets developers write code that gets deployed to that edge network. Now, the company is opening this functionality up to AI-based workloads too by extending support for NVIDIA GPUs and TensorFlow. If computing is to become ubiquitous, this move is a really powerful one. It breaks the paradigm of running expensive AI services in a centralised server or in limited regions, and instead pushes models to Cloudflare’s edge network. 

The Generalist featured a great end-to-end story of UiPath, the enterprise robotic process automation company. 

Samy Bengio, a well-known senior Google machine learning research manager resigned from the company in the wake of the firing of Timnit Gebru and Margaret Mitchell. Bengio oversaw the ethical AI team. He has since joined Apple in a senior role where he will head up a new AI research team.

Here’s a selection of impactful work that caught my eye, grouped into categories:

How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals, Stanford. This study went through the process of aggregating 141 FDA-approved AI device filings to report on how each device was evaluated in the process. They find that many FDA-approved AI devices were only tested on passive data from a handful of sites, which might not provide sufficient patient population coverage to be robust upon deployment. The data is visualized here.

Extraction of organic chemistry grammar from unsupervised learning of chemical reactions, IBM Research Europe and Cambridge, University of Bern. In organic chemistry, we all learn about atom mapping, which is how atoms rearrange during a chemical transformation. It’s not fun and requires lots of rote memorization. In this work, Philippe Schwaller and colleagues apply transformers to this problem. Their model learns the task of atom-mapping by making use of unmapped reactions on the self-supervision task of predicting the randomly masked parts in a reaction sequence without any human labeling or supervision. The training set is reactants and products represented as SMILES strings. You can read more in this blog post. 

Efficient large-scale language model training on GPU clusters, NVIDIA, Stanford, MSR. Masters of scalable high-performance computing, NVIDIA, demonstrate how to use different types of parallelism methods (model, pipeline, and data parallelism) to overcome the scaling challenges with training large language models. Their method produces a 100x increase in the size of the models that can be efficiently trained across thousands of GPUs. The paper describes a 1 trillion parameter model trained on 3,072 GPUs with an almost 50% increase in the per-GPU throughput. 

International control of powerful technology: Lessons from the Baruch Plan for nuclear weapons, Oxford. The drumbeat of ethics and the existential risks of AI is clearly amplified in recent years. History has a lot to teach us about how humanity behaves when it confronts new technologies with extensive capabilities. This paper draws on the development and attempted governance of dual-use nuclear technology for lessons on how to potentially handle general AI. The authors write, “radical levels of cooperation become more feasible…[when] the gains from coordination are tremendous, the losses from failed coordination terrible, and where most actors’ long-term interests are aligned.”

Retrieval augmentation reduces hallucination in conversation, Facebook AI Research. Large language models store knowledge from training data implicity in their model weights. However, they are well known to often “hallucinate” whacky responses that look plausible but are factually incorrect. The authors apply neural-retrieval-in-the-loop for retrieval-augmented generation as a means of resolving these hallucination problems on open-ended dialogue tasks. 

Compositional perturbation autoencoder for single-cell response modeling, Helmholtz Center Munich, Facebook AI Research, Cellarity, Technical University of Munich. Single-cell sequencing is a powerful technique for studying how cells respond to changes in their environment (e.g. in response to drug treatments or genetic perturbations). Systems biology dictates that treating cells with combinations of drugs will result in a hugely complex network of interactions that is hard to explore experimentally. This paper uses machine learning to model how different cell types respond to different drugs across doses and combinations. Using learned embeddings for drugs and cell types, the authors show how their model can predict combinatorial genetic interactions in a way that could facilitate in silico screening for effective drug combinations. You can read more in this blog post and find the model open-sourced here.

Generative AAV capsid diversification by latent interpolation, Dyno Therapeutics, Harvard, Wyss Institute. Adeno-associated virus (AAV) capsids can be used as vectors to deliver gene therapy in the clinic. However, most natural AAV capsids are caught out by pre-existing human immunity. This paper uses machine learning to model segments of the AAV2 capsid and interpolate sequence space to produce diverse and novel capsids that can evade the human immune system. 

Evaluating eligibility criteria of oncology trials using real-world data and AI, Stanford. The recruiting of patients for clinical trials involves setting eligibility criteria that are often overly restrictive and sometimes poorly justified. This means that trials might be excluding patients who could indeed join them, which is a major reason why 86% of clinical trials fail to complete their recruitment on time. This paper evaluates the effect of relaxing specific eligibility criteria on treatment efficacy and cohort size in a real-world population using the Flatiron Health database of 280 cancer clinics in the USA. This produced a doubling of the pool of eligible patients with a modest concomitant 0.05 average reduction in the hazard ratio (the ratio of an event happening in the treatment arm versus the control arm). 

MDETR - Modulated detection for end-to-end multi-modal understanding, NYU and Facebook. This paper presents a joint computer vision and NLP system that can learn to find objects in an image that are mentioned in a free-form text description of that image without having seen that object before. Visual features and text features are fused together into a shared embedding space, concatenated, and then fed into a transformer encoder-decoder that predicts the bounding boxes of the objects and their grounding text. The model is pre-trained on 1.3M text-image pairs and fine-tuned on downstream tasks such as phrase grounding. 

Accelerated antimicrobial discovery via deep generative models and molecular dynamics simulations, IBM Research and Columbia University. This paper presents a computational framework for the targeted design and screening of molecules, which combines attribute-controlled deep generative models and physics-driven simulations. They find, synthesize, and empirically test 20 candidate antimicrobial peptides and find two that are potent and broad-spectrum. 

Skillful precipitation nowcasting using deep generative models of radar, DeepMind. This paper uses generative models to produce realistic and spatiotemporally consistent predictions of rain with lead times from 5-90 mins ahead of real-time. The system was validated by expert forecasters from the UK’s Met Office. 

Evolving reinforcement learning algorithms, Google Research. This paper applies concepts from AutoML to optimize an RL algorithm as a computational graph. In doing so, they discovered algorithms that generalize to more complex environments than those that are hand-designed. 

SambaNova Systems, makers of a specialized AI semiconductor and computing platform, raised a $676M Series D led by SoftBank at a $5B valuation. The company has now raised over $1B since it was founded in 2017. Unlike its rival startups, SambaNova appears to be focused on providing subscription-based AI models to its customers based on its hardware rather than selling datacenter racks or chips for customers to run themselves. 

Scale AI, the data annotation company, raised a $325M Series E co-led by crossover investors Dragoneer, Greenoaks, and Tiger, along with Durable and Wellington. With a roster like this, you can bet the company is prepping to go public. 

Vectra, the cybersecurity AI company for on-premise and cloud-based networks, raised $130M at a post-money valuation of $1.2M led by Blackstone Growth. 

Nuro, the autonomous on-road delivery vehicle company, added Woven Capital (the investment company of Toyota Research Institute) and Chiptole Mexican Grill to its $500M Series C. 

Cresta, a product that improves team performance with live prompts on the best thing to say during every customer interaction, raised a $50M Series B led by Sequoia. The company, which has its roots in the Stanford AI lab, claims that agents convert 20% more sales when using the Cresta system. 

Snorkel, the AI platform emerging from Chris Re’s group at Stanford that introduced labeling functions into the mix, raised a $35M Series B led by Lightspeed. The product has expanded into model training, inference, and collaborative workflows. 

Streamlit, the fastest way to build and share data apps in Python, raised a $35M Series B led by Sequoia. The product is particularly popular amongst data science groups that interface with non-technical teams. 

Comet, the creators of an ML model experimentation platform, raised a $13M Series A led by Scale Venture Partners off the back of “500% year on year growth”, according to the company.

ShareChat and Moj, two AI-driven mobile products for interest-based WhatsAapp group and short video content sharing in India, raised a $502M round at a $2.1B valuation led by Tiger Global. The service is now India’s largest regional social media platform with 160 million MAUs serving 15 Indian languages. 

Signifyd, an AI-based fraud protection software company, raised a $205M round at a $1.34B valuation led by Owl Rock Capital.

Sift, another AI-first fraud prevention company, raised $50M at a valuation over $1B led by Insight Partners.  

Canvas Medical, a new electronic health records software company, raised a $17M Series A led by IA Ventures. The startup also partnered with Anthem, one of America’s largest health insurers. 

Carta Healthcare, a clinical data company that helps physicians navigate patient registries, raised a $17M Series A led by Storm Ventures. 

Veriff, an Estonian identity verification startup that processes video and photos using machine learning, raised a $69M Series B led by IVP and Accel. 

Synthesia, the AI-first video generation platform, raised a $12.5M Series A led by FirstMark.

Tempo, the home fitness hardware product that uses 3D sensors and computer vision to analyze your form and coaches you, raised a $220M Series C led by SoftBank only 9 months after its $60M Series B. The company says it grew 10x in the last year. 

Oxbotica, the Oxford-based developer of a universal on/off-road autonomy solution, raised a £10M Series B from Ocado Group, the UK’s leading online supermarket and warehouse robotics company. As part of the investment, Ocado will “outfit a subset of its delivery vans and warehouse vehicles with data capture capabilities, which may include video cameras, LiDAR, RADAR and other sensing devices. Ocado will make this data available to Oxbotica to train and test its technologies, which will then inform Ocado as to what opportunities exist and where best it might take advantage of these exciting new partnerships.”

Bigeye, the data quality monitoring company, raised a $17M Series A led by Sequoia. The company’s founders previously worked at Uber where they build internal tools for DQM. 

Xwing, a software company providing autonomous capabilities to small propeller aircraft, raised a $40M round at a $400M post-money valuation after demonstrating its first gate-to-gate autonomous commercial cargo aircraft flight. 

Cape Privacy, an enterprise private computation company, raised a $20M Series A led by Evolution Equity Partners. 

Sensei, a Lisbon-based startup providing autonomous store technology to retailers, raised a $6.5M round led by Seaya and Iberis Capital. 

Deep Instinct, an AI-first cybersecurity company, raised a $100M Series D led by BlackRock. 

AutoStore, a Norwegian developer of robotics and software solutions for automated warehouse logistics, entered into an acquisition agreement with SoftBank in which the firm would acquire 40% for $2.8B, valuing AutoStore at $7.7B. AutoStore was founded in 1996 and brought to market a “cube storage automation” product to create very dense warehouse storage. The business now has 20,000 robots deployed in over 600 installations across 35 countries. For those of you familiar with Ocado, the UK online retailer, and robotics company, you’ll notice that AutoStore’s products appear quite related to Ocado’s. In fact, the companies are involved in a lawsuit over IP infringement that kicked off last year. 

Darktrace, the British AI-first cybersecurity company that detects, investigates, and responds to cyber threats within enterprises, filed to list on the LSE. The business was founded in 2013 around maths researchers at the University of Cambridge and has since cemented itself as a global leader. Its ARR grew from $100M in 2018 to $235M in 2020. Customers start with a three-week proof of value and then convert to paid software subscriptions with an average contract ARR of $60k. Its net ARR retention rate is 99%, which is high but not best in class and counts almost 5,000 customers. This listing is a major win for the UK technology industry as it sees a globally recognized AI-first software company poised for long-term growth. 

Nuance, a publicly-traded speech and text recognition company founded in 1992 with almost 7,000 employees, announced its intent to be acquired by Microsoft for almost $20B. This would be Microsoft’s second-largest deal ever, behind its $27B acquisition of LinkedIn in 2016. Nuance has built up a business in the medical domain where its technology processes and transcribes medical notes. Nuance itself has been very acquisitive and defensive over its IP portfolio, which is another reason for Microsoft to take over it to expand its healthcare practice. 

TuSimple, the self-driving trucking company, raised $1B in an IPO that valued the company at $8.5B. The company posted revenues of $1.8M in 2020 and a net loss of almost $200M. 

Plus, another self-driving company, is reportedly in talks to merge with a SPAC run by Hennessy Capital that would value the business over $3B along with $500M in new capital. 

ZebiAI, a Boston-based small molecule discovery company that uses machine learning to improve DNA-encoded library creation, was acquired by Relay Therapeutics, a NASDAQ-listed computational drug discovery company for $85M ($20M cash, $65M in Relay stock). If ZebiAI meets certain commercial targets as part of Relay, shareholders could receive up to an additional $100M. After Haystack’s recent acquisition by insitro, this positions Anagenex (an Air Street portfolio company) as the sole standalone AI-first DEL company. 

Vicarious Surgical, a VR-controlled miniaturized surgical robot, entered into an agreement with a SPAC called D8 Holdings that values the business at $1.1B. The company plans to launch its product on the market in late 2023. It plans to offer surgeons a base station with 9 degrees of freedom per arm, which lets the surgeon make incisions of 1.5cm and see 360 degrees. You can find its investor presentation here.

—

Signing off, 

Nathan Benaich, 9 May 2021

Air Street Capital | Twitter | LinkedIn | State of AI Report | RAAIS | London.AI