Shreyas Cholia's research while affiliated with Lawrence Berkeley National Laboratory and other places

... Minor et al. 76 introduce optical aberrations such as nonuniform brightness, lowfrequency noise, and Gaussian blurring into their simulations of topological defects in liquid-crystal systems. Munshi et al. 77 augmented their converged beam electron diffraction data with the addition of elliptic distortions, random translations of the pattern, incoherent plasmonic background noise, and Poisson shot noise. ...

... Consider the smallest amount of information needed for data to be reusable, the 'Minimum information standard' (see Wikipedia for examples of these minimum standards). An example is the work by ESS-DIVE in establishing a community-centric metadata reporting format [2]. ...

... Such applications have motivated the development of specialized interfaces for remote job submission [63,64] and for managing workloads across systems [65,66,67,68]. The LBNL superfacility project has studied requirements for linking instruments with HPC [69] and proposed an OAuth-based asynchronous API [70] that is similar to our action provider interface. DataFed [71] federates various scientific data stores. ...

... For example, there is no storage limit at the OSF repository and a 50 GB limit per dataset at Zenodo [22]. In rare cases in which datasets exceed these limitations, dataset managers can bundle smaller datasets for easier upload and downstream data reuse [41]. For example, The Climate Modelling Intercomparison Project (CMIP6) data and associated model runs contain petabytes of data, but are divided into smaller 'file sets' for more efficient storage and download [42]. ...

... IRT usually consists of greeting contributors, explaining the project guidelines, and collecting relevant information [6]. Although IRTs were introduced on GitHub in 2016 [7] and developers positively rated the usefulness of IRTs on issue reporting [6], [8], [9], they are rarely utilized, according to our analysis. ...

... As a result, extensive research [1,2] has proposed semi-automation approaches to analyze these datasets, these deterministic methods rely on classical computer vision techniques (e.g., Hough transform, Fourier filtering, segmentation and cross-correlation for similarity measure), which typically require manual hyperparameter tunning and a computation cost for each experiment. Deep neural networks (DNNs) have shown superior performance compared to classical computer vision techniques in most benchmark tasks [10], this led to the emergence of fully automated approaches [4,5] and tools [6,7] for various TEM tasks. In the context of orientation microscopy, ML-based approaches are still falling behind traditional techniques such as template matching [1] or Kikuchi technique [8] when it comes to generalization performance to unseen orientations and phases during training, this is due mainly to the limited amount of experimental data about the studied phenomena for training the models, it is a realistic and practical constraint specially for narrow domain applications where real data is not widely available. ...

... 129,138 Remote execution has been integrated with Jupyter notebooks. [139][140][141] The ability to compute anywhere enables users to leverage specialized computing resources designed for low-cost, distributed, and edge computing. 142 AI systems deployed at experimental facilities support rapid data filtering at the edge. ...

... There is an increasing interest in scientific workflows that incorporate remotely controlled, automated experiments over collections of physical instruments and computing systems. Often, these resources are located at geographically dispersed sites, and they need to be federated over wide-area networks to form ecosystems that seamlessly support these workflows [1], [2]. Recent workflow developments enable the use of Artificial Intelligence (AI) codes both as a part of scientific computations and data analyses, and for orchestrating automated experiments at potentially remote physical instruments. ...

... Jupyter has been used in ML research since its inception as a front-end interactive environment for remote CI. Most highperformance computing (HPC) platforms target Jupyter as their current or next-generation front-end user interface (Thomas and Cholia, 2021). The ML and geospatial data libraries introduced in the previous section can be easily used in Jupyter. ...

... We worked with science teams, including NCEM, ALS, and LCLS data-processing pipelines, to capture common Jupyter workflow patterns that can be deployed in a repeatable manner across multiple projects. We describe a case study from NCEM here as an example [5]. ...