Dear readers,

Welcome to Your guide to AI. Here you’ll find an analytical (and opinionated) narrative covering key developments in AI over May 2021. Before we start, I have two pieces of news!

If you’d like to chat about something you’re working on, share some news before it happens, or have feedback on the issue, just hit reply! I’ll be sending through the audio edition next weekend. 

If you enjoyed the read, I’d appreciate you hitting forward to a couple of friends 🙏

Oxford-based Exscientia is on a roll: the company announced its third molecule is entering clinical trials in collaboration with Sumitomo Dianippon Pharma. The drug, DSP-0038, is to be used for the treatment of Alzheimer’s disease psychosis. What makes the molecule interesting is its selective dual targeting characteristics, which means it hits two receptors while avoiding related and unwanted targets. Exscientia also announced a huge four-year $1.2B deal with Bristol-Myers to discover drugs and exclusively license them to the latter.  

Meanwhile, Recursion announced their first internally developed new chemical entity advanced to investigational new drug-stage. The molecule is a first-in-class small molecule toxin inhibitor that targets C. difficile infections, which usually take hold when you’re on antibiotics. This asset joins the company’s four known chemical entities advancing to phase 2 studies. 

The US Air Force demonstrated the autonomous piloting of a tactical unmanned airplane as part of the Autonomous Attritable Aircraft Experimentation program. The plane could respond to navigational commands, react to geofences, and demonstrate coordinated manoeuvring. Meanwhile, Israel’s IDF claimed the use of AI methods in the conflict with Hamas.  

The Brookings Institute evaluated the planned use of AI in 34 countries to determine their priorities. 

Argo AI, the AV company backed by Ford, unveiled a new long-range Lidar with a claimed range of 400 meters. The product is the outcome of Argo’s acquisition of Princeton Lightwave four years ago. This now means that the major AV contenders, Cruise, Waymo, Aurora, and Argo, all have their home-grown Lidar solutions. Meanwhile, public Lidar stocks (e.g. Luminar, Velodyne, Aeva, Innoviz) are trading far below their SPAC peaks. Does this make the case for Lidar businesses finding a better home within a vertically integrated AV company vs. independent companies? Meanwhile, Tesla allegedly informed the California regulator that it would not produce a fully-self driving vehicle by the end of 2020. Waymo and Cruise both applied for permits to charge for autonomous car rides in SF. 

Apple, Microsoft, Alphabet, Intel, Amazon Web Services and other major buyers of semiconductors created the Semiconductors in America Coalition to lobby the US government to provide the $50B CHIPS for America Act that Biden asked Congress for. On-shoring semiconductor manufacturing capacity is important, but will take a very long time. Today’s supply chain is globally integrated and has many, many steps. 

Geico, the second largest auto insurer in the US, entered into an agreement with London/NYC-based AI-first company Tractable to use their software to speed up vehicle repairs nationwide. After experiencing damage, the car is photographed and Tractable’s software predicts with a high level of confidence if the vehicle is a total loss or is repairable, in addition to which parts need repair and how much it will cost. 

OpenAI launched a $100M startup fund aimed at financing early stage companies that make use of their APIs. The capital comes from Microsoft and other OpenAI partners in a move that looks like customer acquisition and ecosystem development around GPT3 and Azure.

Here’s a selection of impactful work that caught my eye, grouped into categories:

Inferring experimental procedures from text-based representations of chemical reactions, IBM Research. This paper extracts chemical equations and associated actions sequences from patent text using NLP models and uses this dataset to train a model to predict the entire sequence of synthesis steps starting from a textual representation of a chemical equation. 

DriveGAN: Towards a controllable high-quality neural simulation, NVIDIA Research. This paper presents a fully differentiable driving simulator that is learned from real video sequences and their associated actions. This results in a model that can disentangle different components of the simulation without supervision. 

Scaling end-to-end models for large-scale multilingual ASR, Google. This paper explores how model size and data per language affects the ability of ASR from a high-resource language to transfer to lower resource languages. They find that scaling the number of model parameters brings significant performance gains with a concomitant gain in data efficiency and more efficient training cost (measure in TPU days). 

ProtTrans: Towards Cracking the Language of Life’s Code Through Self-Supervised Learning, TU Munich. This paper trains large language models on 393 billion amino acid sequence data to explore whether unsupervised pre-training can learn intrinsic biological structure. Indeed they find this to be true: pre-trained LLMs perform well on per-residue prediction of protein secondary structure, per-protein predictions of protein subcellular localisation, and predicting whether a protein will be membrane-bound or water soluble. Relatedly, there is an open source repo called Bio Embeddings, which lets you quickly predict protein structure and function from sequence. 

High-performance, distributed training of large-scale deep learning recommendation models, Facebook. This paper introduces a 12 trillion parameter model and distributed training system to achieve a 40x speedup on inference time. Indeed, another related paper called Larger-scale transformers for multilingual masked language modelling also by Facebook AI shows that pretraining models with larger capacity (10.7B parameters) yields strong performance on high- and low-resource languages. 

Are pre-trained convolutions better than pre-trained transformers? Google Research. This paper sought to separate the effects of pre-training and architectural advances on large language model performance. By using 8 dataset/tasks for experimentation, they find that CNN-based pre-trained models are competitive and outperform their Transformer counterpart in certain scenarios, albeit with caveats. 

From motor control to team play in simulated humanoid football, DeepMind. This work presents a method for training physically simulated humanoid avatars to play football in a realistic virtual environment. The authors make use of imitation learning, single/multi-agent reinforcement learning and population-based training and a staged task complexity from articulated body control to coordinated goal-directed team behavior. 

SimNet: Learning reactive self-driving simulations from real-world observations, Lyft Level 5. The paper makes use of behavioral cloning from 1000h of data logs. SimNet predicts the next position of each agent independently from the current frame. Agents trained by the model exhibit realistic behaviors across different scenes. 

End-to-end privacy preserving deep learning on multi-institutional medical imaging, TU Munich. This paper describes how securely aggregated federated learning and encrypted inference can be used on medical imaging data (paediatric chest X-rays) to produce a model with classification performance that is on par with locally, non-securely trained models. 

…and after all of this transformer literature, could multi-layer perceptrons replace them?

Bonus round: Kenn Cukier of The Economist just published his latest book, Framers: Human Advantage in an Age of Technology and Turmoil. How? By making use of mental models.

ZOE (an Air Street investment), the personalised nutrition company known for its large-scale microbiome clinical studies, raised a $20M Series B led by Ahren. If you’re based in the US, you can order the product today at www.joinzoe.com

Exscientia, the AI-first clinical stage drug development company, raised a $225M Series D led by SoftBank Vision Fund 2. The Fund is also providing an additional $300M equity commitment that can be drawn at the Company’s discretion. This funding comes on the back of two active clinical trials for drugs designed by Exscientia - a world first. 

Dyno Therapeutics, the adeno-associated viral vector design company focused on gene therapy, raised a $100M Series A led by a16z. 

Einride, the Swedish electric and autonomous freight mobility company, raised a $110M Series B from Temasek, Soros Fund management and others. 

Anthropic, a new AI safety-focused research company formed by alums of OpenAI, raised a $124M maiden financing round. 

Ada Health, the Berlin-based telemedicine company, raised a $90M from by Leaps by Bayer. 

Shift Technology, an AI-first software company built for insurers, raised a $220m Series D led by Advent International at a unicorn valuation. The product offers decision automation across underwriting, subrogation, and compliance. It serves 100 customers in 25 countries.

Mythic, the analog chip company, raised a $70M Series C led by BlackRock and Hewlett Packard Enterprise.  

Tessian, the cybersecurity company focused on human mistakes, raised a $65M Series C led by March Capital. 

EasyMile, the French autonomous shuttle company, raised a €55M Series B led by Searchlight Capital Partners. The company has deployed 180 vehicles in 30 countries to date. 

Cymulate, a cybersecurity company that lets customers run machine-based attack simulations on their networks to discover vulnerabilities, raised a $45M Series C led by One Peak Partners. 

Placer.ai, a location data analytics company, raised a $50M Series B led by angels Josh Buckley, Todd Goldberg and Rahul Vohra. 

ComplyAdvantage, the financial crime prevention company, extended its Series C to $70M with new investment from Goldman Sachs.

Causaly, a software company that helps researchers navigate biomedical research, raised a $17M Series A led by Index. 

Pah Robotics, a company that automates welding in manufacturing plants, raised a $56M Series B led by Addition. 

Plus One, a computer vision control system for logistics robots, raised a $33M Series B led by McRock capital. 

Sprout.ai, an insurance focused startup applying NLP and computer vision to claims automation, raised a $11M Series A led by Octopus Ventures.

Coiled, an MLOps company, raised a $21M Series A led by Bessemer.  

iSIZE, a video compression company, raised a $6.3M round led by Octopus Ventures. 

Lyft Level 5, the self-driving division of Lyft, announced its intention to be acquired by Woven Planet (a Toyota holding company focused on autonomous driving) for $550M. Of this, $200M is upfront and $35M is paid over five years. For Lyft, this also means a saving of $100M in net operating expenses from reduced R&D. To dive deeper into the consolidation of AVs, have a read of this post.

Blue Yonder, the AI-first supply chain software company for retail, manufacturing and logistics customers, was acquired by Panasonic for up to $8.5B. 

Gingo Bioworks, the company that engineers novel organisms to produce useful materials and substances, merged with a SPAC to value the public company at $17.5B post-money. You can find the investor deck here.

WaveOptics, makers of waveguide technology for use in augmented reality headsets, was acquired by Snap for $500M. 

Inivata, a precision medicine liquid biopsy company, was acquired by NeoGenomics for $390M. This follows NeoGenomics’ $25M investment in May 2020. Inivata was formed around technology developed at Nitzan Rosenfeld’s group based in the Cancer Research UK Cambridge Institute (where I did my PhD). 

DeepCube, a software-based inference accelerator, was acquired by Nano Dimension, a 3D printer manufacturer working on additively manufactured electronics. The deal is worth $70M.

Pixel8earth, a 3D mapping company using crowdsourced data, was acquired by Snap for $7.6M. 

—

Signing off, 

Nathan Benaich, 13 June 2021

Air Street Capital | Twitter | LinkedIn | State of AI Report | RAAIS | London.AI