+++ Near-perfect point-goal navigation from 2.5 billion frames of experience +++ Google’s ML-fairness-gym: A Tool for Exploring Long-Term Impacts of Machine Learning Systems +++ Facebook launches PyTorch3D, a 3D deep learning library +++ White House reportedly aims to double AI research budget to $2B +++ Facebook has acquired Scape Technologies, a London-based computer vision startup +++ Avigilon’s Appearance Search Can Tell Schools Where Students Are And Where They’ve Been +++ Google, YouTube, Facebook, LinkedIn, Twitter demand Clearview stop scraping images for facial recognition +++ Quantifying Independently Reproducible Machine Learning +++ RGISTools: Downloading, Customizing, and Processing Time-Series of Remote Sensing Data in R +++ Ignite the Spark: Running Apache Spark on Kubernetes using PySpark +++
Breakthrough — Or So They Say
Near-perfect point-goal navigation from 2.5 billion frames of experience. A team at Facebook AI is announcing that it has created a new large-scale distributed reinforcement learning (RL) algorithm called DD-PPO, which has effectively solved the task of point-goal navigation using only an RGB-D camera, GPS, and compass data. Agents trained with DD-PPO (which stands for decentralized distributed proximal policy optimization) achieve nearly 100 percent success in a variety of virtual environments, such as houses and office buildings. The RL-algorithm lets a robot find its way in an unfamiliar environment without using a map. The algorithm gets a robot to its goal 99.9% of the time along a route that is very close to the shortest possible path, which means no wrong turns, no backtracking, and no exploration. They believe that the agent learns to exploit the statistical regularities in the floor plans of real indoor environments (apartments, houses, and offices) that are also present in their data sets. This breakthrough is partly attributed to Habitat, Facebook’s own open-sourced simulation platform for training embodied AI agents (virtual robots and egocentric assistants) in a highly photorealistic & efficient 3D simulator. The model was also successfully tested with tasks in real-world physical settings using a LoCoBot and Facebook AI’s PyRobot platform. Facebook trained bots for three days inside AI Habitat’s photorealistic virtual mock-up of the interior of a building, with rooms and corridors and furniture. In that time they took 2.5 billion steps—the equivalent of 80 years of human experience. This represented more than six months of GPU-time training, but they completed it in less than three days of wall-clock time with 64 GPUs. The approach is documented in the preprint arXiv:1911.00357.
Tools and Frameworks
Google’s ML-fairness-gym: A Tool for Exploring Long-Term Impacts of Machine Learning Systems. As machine learning becomes increasingly incorporated within high impact decision ecosystems, there is a growing need to understand the long-term behaviors of deployed ML-based decision systems and their potential consequences. Most approaches to understanding or improving the fairness of these systems have focused on static settings without considering long-term dynamics. To address this structural difficulty in the field, Google advocates for the use of simulation as a key tool in studying the fairness of algorithms. Their new framework ML-fairness-gym has now been released on github. The ML-fairness-gym simulates sequential decision making using Open AI’s Gym framework. In this framework, agents interact with simulated environments in a loop. At each step, an agent chooses an action that then affects the environment’s state. The environment then reveals an observation that the agent uses to inform its subsequent actions. In this framework, environments model the system and dynamics of the problem and observations serve as data to the agent, which can be encoded as a machine learning system.
Facebook launches PyTorch3D, a 3D deep learning library. One of the reasons 3D understanding with deep learning is relatively underexplored compared with 2D understanding is because 3D data inputs are more complex with more memory and computation requirements, whereas 2D images can be represented by simple tensors. PyTorch3D, just released on github, now provides efficient, reusable components for 3D Computer Vision research with PyTorch, such data structures for storing and manipulating triangle meshes, efficient operations on triangle meshes (projective transformations, graph convolution, sampling, loss functions), and a differentiable mesh renderer.
Business News and Applications
White House reportedly aims to double AI research budget to $2B. According to Reuters, The White House will propose a big jump in U.S. government spending on artificial intelligence (AI) and quantum information sciences research and development in its 2021 budget proposal, two people briefed on the plan said. The Trump administration wants Congress to double spending on AI R&D funding from $973 million to nearly $2 billion by 2022 and to double spending on quantum information sciences spending to $860 million within two years. The figures do not include some Defense Department research programs. Officials say the jump is aimed at keeping pace with China’s efforts.
Facebook has acquired Scape Technologies, a London-based computer vision startup. Scape Technologies, a London-based computer vision startup working on location accuracy beyond the capabilities of GPS, has been acquired by Facebook, according to an article on TechCrunch. Founded in 2017, Scape Technologies was developing a “Visual Positioning Service” based on computer vision which lets developers build apps that require location accuracy far beyond the capabilities of GPS alone. The technology initially targeted augmented reality apps, but also had the potential to be used to power applications in mobility, logistics and robotics. More broadly, Scape wanted to enable any machine equipped with a camera to understand its surroundings. The acquisition will support Facebook’s investment in next generation platforms, including VR and AR.
Avigilon’s Appearance Search Can Tell Schools Where Students Are And Where They’ve Been. As mass shootings at US schools increase in frequency, more school administrators across the US are turning to artificially intelligent surveillance tools in an attempt to beef up school safety. Recode takes a closer look at one such solution, the AI-based software Appearance Search by Avigilon. Appearance Search allows school administrators to review where a person has traveled throughout campus — anywhere there’s a camera — using data the system collects about that person’s clothing, shape, size, and potentially their facial characteristics, among other factors. It also allows security officials to search through camera feeds using certain physical descriptions, like a person’s age, gender, and hair color. So while the tool can’t say who the person is, it can find where else they’ve likely been. For some, this raises big concerns …
Google, YouTube, Facebook, LinkedIn, Twitter demand Clearview stop scraping images for facial recognition. According to recent news coverage (see e.g. here, here, here, and here) most of the companies went so far as to send cease-and-desist letters to Clearview AI, telling the startup that it can no longer scrape videos from its platform. Clearview AI came under scrutiny this year after reports that it had been scraping videos of people on social media platforms and then adding faces to its database. Its facial recognition technology can use an image of a person and then find more links where that face has appeared. The company has claimed that it has more than 3 billion images in that database, images that were scraped from Facebook, YouTube, Twitter and other websites. Clearview’s technology has so far been used by local police departments in the U.S. as well as the FBI and the Department of Homeland Security. A comprehensive feature in The New York Times in January had brought Clearview to general attention.
Publications
Quantifying Independently Reproducible Machine Learning. An article in the Gradient by Edward Raff where he summarizes his earlier NeurIPS 2019 paper (arXiv:1909.06674). Raff attempted to re-implement 255 papers published from 1984 until 2017, recording features of each paper, and performing statistical analysis of the results. For each paper, he did not look at the authors code, if released, in order to prevent bias toward discrepancies between code and paper. Some of this findings: (1) theory-oriented papers, with many derivations and equations, don’t lead to better reproducibility; (2) open sourcing of code is at best a weak indicator of reproducibility; (3) surprisingly, having detailed pseudo code is just as reproducible as having no pseudo code; (4) also surprisingly, creating simplified example problems do not appear to help with reproducibility.
Understanding Data Scientists’ Use of Interpretability Tools for Machine Learning. A lot of research is currently being performed on the interpretability of machine learning models. Interpretability tools such as SHAP have been designed to help data scientists and machine learning practitioners better understand how ML models work. However, there has been little evaluation of the extent to which these tools achieve this goal. A conference paper is now stating research findings that indicate that data scientists often over-trust and misuse interpretability tools. Furthermore, few of the study’s participants were able to accurately describe the visual output by these tools.
Tutorials
RGISTools: Downloading, Customizing, and Processing Time-Series of Remote Sensing Data in R (arXiv:2002.01859). There is a large number of data archives and web services offering free access to multispectral satellite imagery. Archives and web services differ in their protocols, formats, and data standards, which are barriers to combine datasets. This article presents RGISTools, an R package to create time-series of multispectral satellite images from multiple platforms in a harmonized and standardized way. The package functionalities are described and then applied in an extended case study.
Ignite the Spark: Running Apache Spark on Kubernetes using PySpark. In this primer, published on Towards Data Science, you are first going to learn a little about how Apache Spark’s cluster manager works and then how you can run PySpark within a Jupyter notebook interactively on an existing Kubernetes (k8s) cluster. After completing this article, you should be able to develop Spark applications on any Kubernetes cluster with confidence as well as have a deeper understanding of how the two are wired together.