
Marcin LawendaPoznańskie Centrum Superkomputerowo-Sieciowe · Data Processing Technologies Division
Marcin Lawenda
PhD
HiDALGO2 Project Coordinator (hidalgo2.eu)
About
145
Publications
12,348
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
316
Citations
Introduction
Dr. Marcin Lawenda (M) completed his M.Sc. in Computer Science, specializing in Parallel and Distributed Computation, at Poznan University of Technology in 2000. He earned his Ph.D. from the same institution in 2006. With over 25 years of experience, he has been employed at the Poznan Supercomputing and Networking Center. His research focuses on parallel and distributed processing, scheduling, and Cloud technologies, particularly within the realm of applied sciences.
Education
September 2000 - July 2006
October 1998 - June 2000
October 1995 - June 1998
Publications
Publications (145)
In this paper we study multi-installment divisible load processing in a heterogeneous distributed system with limited memory.
Divisible load model applies to computations which can be arbitrarily divided into parts and performed independently in parallel.
The initial waiting for the load may be shortened by sending many small chunks of load instead...
Accurate digital twinning of the global challenges (GC) leads to computationally expensive coupled simulations. These simulations bring together not only different models, but also various sources of massive static and streaming data sets. In this paper, we explore ways to bridge the gap between traditional high performance computing (HPC) and data...
The Natural History Collections of Adam Mickiewicz University (AMUNATCOLL) in Poznań contain over 2.2 million specimens. Until recently, access to the collections was limited to specialists and was challenging because of the analogue data files. Therefore, this paper presents a new approach to data sharing called the Scientific, Educational, Public...
The article presents optimization techniques for two Python-based large-scale social sciences applications: SN (Social Network) Simulator and KPM (Kernel Polynomial Method). These applications use MPI technology to transfer data between computing processes, which in the regular implementation leads to load imbalance and performance degradation. To...
Forced displacement of people worldwide, for example, due to violent conflicts, is common in the modern world, and today more than 82 million people are forcibly displaced. This puts the problem of migration at the forefront of the most important problems of humanity. The Flee simulation code is an agent-based modeling tool that can forecast popula...
The current document is the update of the Data Management Plan of the HiDALGO2 project (HiDALGO2 DMP), which was issued as the deliverable D1.5 of the project.
In the current document, first the methodology is revised. We added some attributes to the dataset description tables that better describe metadata information, storage, data sharing and se...
This deliverable, D2.5, is the second one of the series of documents (D2.4, D2.5, D2.6) on “Infrastructure Provisioning, Workflow Orchestration and, Component Integration” that reports on the updates, since the first document of the series, D2.4, on different topics reported therein:
· The holistic HiDALGO2 platform of computing and services integr...
HiDALGO2 focuses on the development of high-fidelity simulations to address critical global challenges. This document builds on the foundations of D5.3 [1] by highlighting the research advances achieved within the HiDALGO2 project between M11 and M23 across the five pilot applications: Urban Air Project (UAP), Urban Buildings (UB), Renewable Energy...
Ensemble runs involve executing multiple simulations or models either in parallel or sequentially, often with varying input data, initial conditions, or model parameters. These ensemble scenarios can be categorized in two main ways:
- Parallel Independent Runs (Type 1): Executing several instances of the same model or simulation simultaneously, but...
This work explores the importance of performance-energy correlation for CFD codes, highlighting the need for sustainable and efficient use of clusters. The prime goal includes the optimisation of selecting and predicting the optimal number of computational nodes to reduce energy consumption and/or improve calculation time. In this work, the utilisa...
This work is concerned with the evaluation of the performance of parallelization of learning and tuning processes for image classification and large language models. For machine learning model in image recognition, different parallelization methods are developed based on different hardware and software scenarios: simple data parallelism, distribute...
Simulations have become a prevalent method in the scientific community for researching climate, environmental, and social phenomena. These simulations aid in understanding how various elements such as air, pollution, smoke, and heat disperse in complex spatial environments. Computational Fluid Dynamics (CFD) is a widely used application for conduct...
This deliverable provides an overview of the components, that make up the core foundation of the HiDALGO2 ecosystem and support the project’s use cases in achieving their scientific, technical and environmental goals. It builds upon the work of previous deliverables in HiDALGO2, as well as of D2.1, “Requirements Analysis and Scenario Definition”. I...
As part of the HiDALGO2 project, D5.6 marks a critical milestone in our pursuit of harnessing High-Performance Computing (HPC) to address global challenges through our pilot applications. This deliverable documents the initial outcomes of our pilot implementations, setting a foundation for subsequent advancements and integration strategies. Coverin...
This deliverable documents the initial outcomes of synergies and coupling scenarios identified between the pilots, extending the capabilities of HiDALGO2 use cases.
The discussion started initially with the work in deliverables D4.1 and D5.3. Covering the period up to Month 18, D5.1 encapsulates the progress made in enhancing the capabilities and i...
The main objective of the HiDALGO2 project is to explore, develop, and implement methods and techniques that utilize exascale HPC resources to address current environmental challenges such as air pollution, the use of renewable energy, sustainable building practices, and the prevention and mitigation of wildfire effects
amongst others. The physical...
Deliverable D4.3 develops foundational methodologies for integrating High-Performance Data Analytics (HPDA) and Artificial Intelligence (AI) into the workflows of pilot applications addressing global challenges. This document provides a blueprint for enhancing performance through advanced data analytics and AI across various stages of application d...
The training initiatives within HiDALGO2 are designed to address and bridge competence gaps between High-Performance Computing (HPC) experts and scientists working on Global Challenges within the current offerings of our collaborating partners. HiDALGO2’s primary objective is to address the skills gap and better support Global Challenges scientists...
A thorough understanding of the development direction of the technologies on which modern HPC systems are based is the key to building efficient applications that take full advantage of the offered hardware capabilities.
This deliverable outlines the first approach of HiDALGO2 implementations to innovative HPC technologies. Particularly, the report...
Urgent computing is a specialized type of computing that prioritizes and expedites the execution of time-critical tasks. Unlike traditional batch computing, where jobs are queued and executed in a sequential order, urgent computing provides immediate access to computational resources for critical applications that require timely analysis, decision-...
This deliverable document provides an overview of the dashboard component and the connected services of the HiDALGO2 ecosystem and supports the project’s users and stakeholders for a one-gate architecture to reach HiDALGO2 services. It builds upon the work of previous deliverables such as D2.4 “Infrastructure Provisioning, Workflow Orchestration an...
This deliverable provides a report on the HiDALGO2 project’s dissemination and communication activities that aim to raise awareness among the industrial and research community, disseminate innovative project results to various communities of the HPC, HPDA, and AI and interact with related groups. The report serves as a comprehensive document outlin...
A report that defines the benchmarking methodology that will be followed within the
HiDALGO2 project and presents the initial findings of the benchmarking of the HiDALGO2 pilots on various EuroHPC JU systems.
This work outlines an adaptation methodology for selected large-scale applications to modern HPC systems, including vendors of traditional x86-64 and ARM-based CPUs. This training investigates reducing performance bottlenecks and overcoming memory limitations to improve overall performance. In particular, the proposed methodology is based on HPC re...
A report on the initial progress on the technical infrastructure of the HiDALGO2 project.
Achieving HiDALGO2’s objectives requires the use of advanced environmental simulation systems to estimate and predict the evolution of the different pilots defined in the project.
Due to the complexity of these systems, it is necessary to use HPC resources (such as the EuroHPC supercomputers) to enhance simulation performance. Additionally, employi...
Zbiory historii naturalnej (NHC) odgrywają kluczową rolę w badaniu różnorodności i zmienności organizmów. Krajowe centra informacyjne przeżywają obecnie renesans w wyniku rewolucji informatycznej, w tym rozwoju Systemu Informacji Geograficznej (GIS). Ten renesans wyraża się przede wszystkim w otwartym dostępie do coraz większej ilości cyfrowych dan...
In pursuit of addressing global challenges and to provide stakeholders and decision makers tools that allow to mitigate tragic consequences, HiDALGO2 has embarked on a mission to integrate, implement, and optimize selected environmental use cases using advanced simulation systems. These endeavours, owing to their intricacy, require the utilization...
Nature museums and university natural collections deal with the deposition and conservation of numerous representatives of flora and fauna. The work related to the maintenance of the collection is time-consuming and expensive, often exceeding the financial capacity of a given institution. It is very reasonable for subsidies allocated for this purpo...
This document is the deliverable D1.5 “Data Management Plan” of the HiDALGO2 project. The document describes the methodology and the initial status of the processes for data management in the project. The data management plan, based on the HiDALGO2 Grant Agreement, implements the official recommendation by the European Commission for data managemen...
This document is a deliverable of the HiDALGO2 project, funded by the European High-Performance Computing Joint Undertaking (‘granting authority’), under the powers delegated by the European Commission (‘European Commission’), under the Call “HORIZON-EuroHPC JU-JU-2021-COE-01 (CENTRES OF EXCELLENCE FOR HPC APPLICATIONS)”, and proposal No 101093457....
This paper describes a project aimed at digitizing and openly sharing the natural history collections (AMUNATCOLL) of the Faculty of Biology at Adam Mickiewicz University in Poznań (Poland). The result of this project is a database (including 2.2 million records) of plant, fungal and animal specimens, which is available online via the AMUNATCOLL po...
In this work, we focus on the KPM (Kernel Polynomial Method) approach
to computing the approximate spectrum of eigenvalues of a graph.
The developed KPM code is utilized to validate the AI Social Network simulator,
which simulates the flow of messages through the graph of a social network
(https://hidalgo-project.eu/use-cases/social-networks). The...
The paper describes the interfaces implemented in the AMUNATCOLL IT system, which enable access to and explorationand manipulation of data available in the database containing unique natural collections from the Faculty of Biology of Adam Mickiewicz University in Poznań (FBAMU). Data can be accessed using the two available interfaces: graphical and...
This paper describes the procedures and operational aspects related to the proper storage and handling of taxonomic, biogeographic and ecological data of biological specimens digitised under the AMUNATCOLL project. In the introductory phase of this process, the definition of the metadata is carried out, which is the formal handler of the structure,...
For the most part, this report focuses on the evaluation and improvement of the performance of both simulation and data analysis applications as well as data management and visualization solutions. This topic appears throughout most of the chapters, presenting this issue from various perspectives. In general, the method of presentation of the analy...
Our benchmark suite complements mature generic benchmark suites such as HPL, HPCG, or SPEC, and benchmark suites for global systems science applications such as CoeGSS benchmark suites in several way. It introduces a simple and portable methodology for collecting and reporting benchmark results, a wide set of automation scripts, as well as a rich s...
In the HiDALGO Centre of Excellence CoE (duration Nov 2018 - Feb 2022), about 60 scientists from different disciplines worked together. A total of 13 partner institutions from seven countries were involved. One of the main challenges was to create a working environment where the different disciplines interact in an optimal and constructive way. Con...
Document represents the final report of WP6 concluding its developments. This report presents (i) the final set of requirements and their KPIs, (ii) the Artificial Intelligence (AI) enabled use case workflows, as well as highlights (iii) the final outcomes of the integration process. It should be noted that all respective objectives have been succe...
This deliverable presents the final version of the HiDALGO Portal and its operations. First of all, it presents the main portal features, the selected architecture, and changes from the second version of this deliverable are pointed out. Noteworthy changes affected the following services and tools:
• Single-Sign-On,
• workflow orchestrator, the cha...
Cyfrowa baza zbiorów przyrodniczych Wydziału Biologii Uniwersytetu im. Adama Mickiewicza w Poznaniu.pdf
We discuss optimization techniques implemented on HiDALGO's use case applications based upon both CPU and GPGPU processing.
The report provides information on new promising technologies, which appear on the market and could have significant influence on the functionality and performance of HiDALGO solutions. Furthermore, the HiDALGO benchmark tests are delivered based on the available systems.
This paper introduces two-dimensional implications. In the first place, it ma...
We focus on High Performance Computing (HPC) and High Performance Data Analytics (HPDA) processing methodologies implemented and used in the HiDALGO project in order to conduct computation within multi-stage use case specific workflows.
An introductory talk provides a view of HPC and HPDA from the HiDALGO perspective listing the infrastructure and...
This document provides the initial strategies for optimising applications and implementing novel algorithms and methods. In particular, initial strategies for coupling applications in conjunction with WP4 are provided.
In respect of our approach to High Performance Data Analytics (HPDA), this document does not aim to introduce and test applications...
With over 79 million people forcibly displaced,
forced human migration becomes a common issue in the modern
world and a serious challenge for the global community. The
Flee is a validated agent-based social simulation framework for
forecasting the population displacements in the armed conflict
settings. In this paper, we present two schemes to para...
Introduction to Workshop on High-performance Data Analytics co-organized by ENCCS and HiDALGO. It introduces HIDALGO project by presentation of project's motivation and scope along with use case description. HPDA methodology and initial findings are presented in respected various methods tested on Spark framework.
This document aims at describing the implementation of the second release of the HiDALGO Portal (to be renamed as the Global Challenges Portal), which gives access to HiDALGO services in a simple way, as a one-stop-shop. Such solution consists of a set of tools covering several aspects useful for HiDALGO stakeholders, like training, execution of si...
Deliverable 3.1 focuses on the HPC aspects of HiDALGO, and in particular, sets the guidelines of the HPC benchmarking methodology followed within the project. HiDALGO aims to follow a systematic, reproducible, and interpretable methodology for collecting and storing benchmarking information from the HiDALGO Pilots, to serve their systematic develop...
The common aim of such events is to raise mutual awareness amongst the GC and the HPC/HPDA communities. Subsequently, this deliverable analyses the current state-of-the-art of HiDALGO training activities and innovation workshops, and defines the necessary roadmap for future events. Other equally important aspects of T7.3 that are described in this...
Presentation "Hardware and Software Co-Design Aspects in Social Science Simulations" was given at HiPEAC 2021 Conference, 5th Heterogeneous Hardware & Software Alliance Workshop, Future Generation of Heterogeneous Computing
HiDALGO’s strategy for external community building includes an introduction to HiDALGO’s offerings, a list of the main target groups for building a community around HiDALGO, a strategy for marketing and collaborations, a training concept and a short characterisation of past and planned events. HiDALGO’s offerings include networking, consulting, eas...
Understanding major global challenges (GCs) as well as their underlying parameters is a vital issue in our modern world. The importance of assisted decision making by addressing global, multi-dimensional problems is more important than ever. To predict the impact of global decisions with their dependencies, we need an accurate problem representatio...
Presently, development and optimization activities within HiDALGO project are conducted considering already available hardware and software solutions. However, computer technology constantly develops and it becomes equally important to use the recent achievements offered in this field. That drives us to the main purpose of this document, which is t...
Accurate digital twinning of the global challenges (GC) leads to computationally expensive coupled simulations. These simulations bring together not only different models, but also various sources of massive static and streaming data sets. In this paper, we explore ways to bridge the gap between traditional high performance computing (HPC) and data...
This document shows how specific HPC requirements arising when dealing with GC can be collected in order to come up with such a curriculum. Additionally to the collection of requirements, the quality of the HiDALGO curriculum depends on both didactic and technical best practices in training. At the same time, training events are opportunities to ex...
HiDALGO support system adapts IT Service Management (ITSM) framework to provide an overview of the system and defines the support concepts to provide a guideline in the support provisioning. The support concept details the reasoning behind the design of multiple sub-supporting systems, the selection of multiple supporting tools, best practices in t...
HiPEAC Conference
HPC and Big Data Technologies for Global Systems
Interactive Workshop and Hands-on Session
The document presents the features available, comparing the current implementation with the original plans. Then, for each main feature, the document presents how the feature was implemented, it describes the available APIs for each component (both graphical and REST APIs) and provides information about how to use the features. In the case of REST...
We have worked on different tasks and objectives. On one side, our work focused on internal community building. On the other side, we used our communication channels to disseminate our results to a wide audience, with a special focus on our main stakeholder groups. Furthermore, we planned and conducted ample event management and collaboration activ...
This document provides the initial strategies for optimising applications and implementing novel algorithms and methods. In particular, strategies for coupling applications in conjunction with WP4 will be provided.
Along with this document we are delivering basic knowledge about HPDA applications and their capabilities by providing performance test...
Summary on feedback about “Report on The Retrieval, Provision And Analysis Of Policy And Cost-Related Information / Indicators” prepared under e-IRGSP5 WP4 activity
HiDALGO’s target is the definition of a generic, systematic, reproducible, and interpretable methodology for collecting benchmarking information from the HiDALGO applications, and a systematic way of storing benchmarking results. To achieve that, this deliverable studies the existing HiDALGO infrastructure, surveys available tools, and draws from b...
This deliverable reports on the initial status of the pilot applications in HiDALGO. This includes the three core applications (migration, urban air pollution and social media), as well as models and data sources that are planned to be coupled in to these applications. It serves to provide basic awareness of the HiDALGO applications to the consorti...
Metrics/indicators and in particular Key Performance Indicators (KPIs) become more and more popular tools in many areas of functioning of various activities. They show their usefulness primarily in monitoring the progress of activities, which are crucial to achieve specific goals and improve the performance of endeavour process. It is good to recal...
The document describes the hardware, software and services offered by the three supercomputing centres HLRS, PSNC and ECMWF. The main focus is given on using the already available resources including the process of accessing these resources.
The HPC resources require specialized expertise to use them efficiently and thus, the deliverable provides t...
Dissemination and community building are key aspects of the HiDALGO project. Our results and tools have to be communicated internally as well as to our stakeholders. With the dissemination activities various communities are addressed, who may benefit from the project.
This deliverable describes the foundation, on which dissemination and community...
In this report, we provided a summary of the work conducted in the e-IRGSP5 in the area of collection, aggregation and publication of policy and cost-related information/metrics/Key Performance Indicators (KPIs). In the first place, we have defined the audience to which it is addressed, being mainly funding agencies and policy makers in the area of...
This document is foreseen as a continuation of the previous deliverable D3.6 where achievements were addressed in implementing services specified in the deliverable D3.2 as far as they were realized to be integrated in release 3of the portal. Moreover, it addresses other WP3 achievements in the implementation of methods, tools and mechanisms forese...
With this deliverable we continue to capture the status of new MTMs developed and planned by CoeGSS in WP3 (tasks T3.1–T3.6) and how WP3 has been utilized by the pilots in WP4. We proceed through the main areas covered by the six tasks of WP3, followed by a chapter on how WP3 and WP4 interact.