Enric Tejedor's research while affiliated with CERN and other places

... Python is increasingly relevant for the HEP community, and particularly so in the area of analysis. RDataFrame, like all of ROOT, offers dynamically generated Python bindings via PyROOT ( [10]), while its internal event loop remains in C++: this offers greater opportunities for performance optimizations and allows multi-thread parallelism that would be more complicated in Python due to its Global Interpreter Lock (the GIL). At the same time, the Python ecosystem has developed tools to generate efficient machine code from Python code to speed up numerical applications; the most interesting such tool for our purposes is Numba ( [11]), which is able to compile Python functions (that restrict themselves to use a specific subset of the language) to optimized machine code. ...

... Awkward RDataFrame [13] uses the C++ header-only libraries to simplify the process of justin-time (JIT) compilation in ROOT [14]. The ak.from_rdataframe [15] function converts the selected ROOT RDataFrame [16] columns as native Awkward Arrays. The templated headeronly implementation constructs the Form from the primitive data types [17]. ...

... SeQuiLa has been successfully deployed to both popular managed Hadoop services like Google Dataproc (utilized also in Krissaane et al., 2020) and managed Kubernetes services like Google Kubernetes Engine (GKE), Azure Kubernetes Service, or Amazon Elastic Kubernetes Service. Figure 3 presents an exemplary setup on GKE using the spark-on-k8s-operator and SeQuiLa application defined as a Kubernetes Custom Resource Definition. This architecture was suggested in Castro et al. (2019) as a preferred Apache Spark deployment scenario for scaling data analytics workloads and enabling efficient, on-demand utilization of resources in the cloud infrastructure. More detailed information on setup and corresponding Terraform modules can be found in the dedicated GitHub repository (https://github.com/biodatageeks/sequila-cloudrecipes). ...

... A later study overcame this issue, but did not achieve higher scaling when using available CERN storage facilities [29]. An example of good scalability was provided by researchers of the TOTEM experiment at CERN, with a first approach at distributing a ROOT application over Spark resources in a cloud [30]. The presence of Spark in the HEP community has become relevant enough that CERN has invested in specific infrastructure to support Spark analysis workflows [31]. ...

... With this setup, TOTEM scientists validated the obtained physics results and demonstrated that it is possible to reduce the time required for the final steps of their analysis by a factor 280x when compared to single core execution (i.e., from 8 hours to less then 2 minutes). We invite the interested reader to refer to [14] for further details. ...

... The RNTuple classes make use of templates, such that for simple types (e.g., vectors of floats) that are known at compile time, the compiler can inline a fast path from the highest to the lowest layer without additional value copies or virtual calls. The event iteration layer provides the user-facing interfaces to read and write events, either through RDataFrame [4] or as hand-written event loops. The user interface is presented in more detail in Section 3. ...

... When processing large volumes, sometimes the time spent on reading or writing data comes to the fore. New possibilities for parallelizing these processes are actively used in the optimization of HEP software [5][6][7]. ...

... Such classes are currently available for TMVA [11] (through the TMVA::Experimental::RReader class included in ROOT), lwtnn [12], Tensorflow [9] (through the C API), PyTorch [13] (using TorchScript), and ONNX-Runtime [14]. Most of these support multiple input nodes, with potentially different types, and are thread-safe, such that implicit multithreading [15] can be used. The performance may in the future be improved by evaluating on a batch instead of a single example at a time. ...

... The containerization ecosystem has become so matured, presenting a whole lot of its orchestrators such as Docker Swan, Kubernetes (k8s), Marathon, and Amazon container engine. Google Container Engine (GKE), and Azure container service, Kata [7], Piparo et al. [8], Sanchez [9], Hoque et al. [10] and Augustyn and Warchal [11]. The test bed experiments in the paper made use of Kata-containers. ...