No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
BOINC: A System for Public-Resource Computing and Storage
Abstract
BOINC (Berkeley Open Infrastructure for Network Computing) is a software system that makes it easy for scientists to create and operate public-resource computing projects. It supports diverse applications, including those with large storage or communication requirements. PC owners can participate in multiple BOINC projects, and can specify how their resources are allocated among these projects. We describe the goals of BOINC, the design issues that we confronted, and our solutions to these problems.
... At present, there is a growing need for computational power for scientific research, machine learning and graphics rendering in large ecosystems. This area has evolved from projects like BOINC [27], which relied on the goodwill of users to solve problems like DNA folding with their spare CPU cycles [28]. Some algorithms, such as machine learning and deep learning algorithms, and other sophisticated solutions are raising demands for high-performance hardware and more bandwidth to address the needs of enterprises and businesses in minutes [29]. ...
... Many blockchain dApp systems are struggling with challenges around identity. Some systems such as ZCash 27 and Monero 28 try to hide the identity of users and transactions. There has also been recent work to add the ability for anonymity on top of existing blockchains, particularly in use-cases like Initial Coin Offerings (ICOs), where money is being fund-raised through smart contracts and regulatory bodies require the Know Your Customer (KYC) and the Anti Money Laundering (AML) checks [37] without giving up the identity of the contributors to the entire global network. ...
... It is also desirable that the potential project has an active community of developers (internal and external) -which may be measured by metrics such as contributors, code commits and branches. Depending on the dApp, one may look for a blockchain technology that supports smart contracts and some form of scalable payments such as payment channels, as well as the economic model 27 of the dApps being built on top, and has support for the correct programming languages for the project. To illustrate some of these considerations, the Bitcoin project and the Ethereum project are compared, however, any other project could be subjected to a similar comparison and analysis when selecting an appropriate technology for implementation. ...
Blockchain technology has attracted tremendous attention in both academia and capital market. However, overwhelming speculations on thousands of available cryptocurrencies and numerous initial coin offering (ICO) scams have also brought notorious debates on this emerging technology. This paper traces the development of blockchain systems to reveal the importance of decentralized applications (dApps) and the future value of blockchain. We survey the state-of-the-art dApps and discuss the direction of blockchain development to fulfill the desirable characteristics of dApps. The readers will gain an overview of dApp research and get familiar with recent developments in the blockchain.
... While the problem of using other nodes for executing the tasks of a job has been widely used in distributed computing [3,4], cyber foraging [5], and crowdsourcing [8],the existing work on task allocation cannot be simply adopted for mobile cloud computing. First, the task allocation method must take into account the selfish behavior of mobile nodes by providing incentives for them. ...
... Many existing works in distributed computing (e.g.,SETI@ Home [4], BOINC [3], and cyber foraging [5]) have proposed using other nodes for executing the tasks of a job. However, all of these existing works primarily assume altruistic behavior in the distributed computing environment and do not carefully incentivize resource sharing. ...
We propose a game theoretic framework for task allocation in mobile cloud computing that corresponds to offloading of compute tasks to a group of nearby mobile devices. Specifically, in our framework, a distributor node holds a multidimensional auction for allocating the tasks of a job among nearby mobile nodes based on their computational capabilities and also the cost of computation at these nodes, with the goal of reducing the overall job completion time. Our proposed auction also has the desired incentive compatibility property that ensures that mobile devices truthfully reveal their capabilities and costs and that those devices benefit from the task allocation. To deal with node mobility, we perform multiple auctions over adaptive time intervals. We develop a heuristic approach to dynamically find the best time intervals between auctions to minimize unnecessary auctions and the accompanying overheads. We evaluate our framework and methods using both real world and synthetic mobility traces. Our evaluation results show that our game theoretic framework improves the job completion time by a factor of 2-5 in comparison to the time taken for executing the job locally, while minimizing the number of auctions and the accompanying overheads. Our approach is also profitable for the nearby nodes that execute the distributor's tasks with these nodes receiving a compensation higher than their actual costs.
... This leads to the interesting issue of resource assignment or how to allocate a set of nodes for a given application. Examples of targeted platforms for such a service are telecommunication platforms, where some set of peers may be automatically assigned to a specific task depending on their capabilities, testbed platform such as Planetlab [2], or desktop-grid-like applications [1]. ...
... Each person is represented as a small cross on these axes. 1 Each slice is represented as an oval. The slice S 1 = S 0, 1 2 contains the five shortest persons and the slice S 2 = S 1 2 ,1 contains the five tallest persons. ...
Peer to peer (P2P) systems are moving from application specific architectures to a generic service oriented design philosophy. This raises interesting problems in connection with providing useful P2P middleware services that are capable of dealing with resource assignment and management in a large-scale, heterogeneous and unreliable environment. One such service, the slicing service, has been proposed to allow for an automatic partitioning of P2P networks into groups (slices) that represent a controllable amount of some resource and that are also relatively homogeneous with respect to that resource, in the face of churn and other failures. In this report we propose two algorithms to solve the distributed slicing problem. The first algorithm improves upon an existing algorithm that is based on gossip-based sorting of a set of uniform random numbers. We speed up convergence via a heuristic for gossip peer selection. The second algorithm is based on a different approach: statistical approximation of the rank of nodes in the ordering. The scalability, efficiency and resilience to dynamics of both algorithms relies on their gossip-based models. We present theoretical and experimental results to prove the viability of these algorithms.
... For example, MapReduce is able to process massive data sets by distributing the load over a large number of computers where data are located [13]. Similarly, the Berkeley Open Infrastructure for Network Computing (BOINC [1]) offers a generic infrastructure to disseminate various tasks requiring multiple types of resources (CPU, memory, bandwidth, storage,. . . ) over a large pool of heterogeneous devices (computers, game consoles,. . . ). ...
... For easing the display of the results, the performance of class-i jobs is quantified by the inverse of the mean response time, 1 T i . This metric also happens to be the mean service rate γ i received by the jobs of class i, as we have ...
Understanding the performance of a pool of servers is crucial for proper dimensioning. One of the main challenges is to take into account the complex interactions between servers that are pooled to process jobs. In particular, a job can generally not be processed by any server of the cluster due to various constraints like data locality. In this paper, we represent these constraints by some assignment graph between jobs and servers. We present a recursive approach to computing performance metrics like mean response times when the server capacities are shared according to balanced fairness. While the computational cost of these formulas can be exponential in the number of servers in the worst case, we illustrate their practical interest by introducing broad classes of pool structures that can be exactly analyzed in polynomial time. This extends considerably the class of models for which explicit performance metrics are accessible.
... Over the years, especially in West African region, researchers' and stakeholders' confidence in the use of climate models is increasing. This is due to improvements in nearly all aspects of climate models' fidelity and skill, as well as more detailed understanding of the degree of fidelity and skill (Mariotti et 2 Consequently, information from climate models are extensively being used by the region's policy makers and various socio-economic sectors (e.g., water resources management, agriculture, engineering, environmental management, health, insurance, researchers, etc.) either for risk management or for day-to-day, season-to-season or long-term strategies and planning (Tall et al., 2012;Niang et al., 2014;Nkiaka et al., 2019). Proliferation of climate models calls for caution among researchers and stakeholders. ...
... (CPDN: https://www.climateprediction.net) server facility hosted by the University of Oxford (Anderson, 2004 3 system is to be used to understand changes in extreme weather over West Africa, then it is pertinent to evaluate the performance of the w@h2 simulations over the region. ...
Weather and climate forecasting is extremely important for improving the socio-economic well-being of the West African region. It safe-guards the region from weather and climate related disasters. Hence, utilization of products from climate models are being encouraged and have therefore become essential tools and life-savers, in spite of the fact that climate models do not fully comply with attributes of forecast qualities - RASAP: reliability, association, skill, accuracy and precision. This paper thus quantitatively evaluates, in comparisons to CRU and ERA5 datasets, the RASAP compliance-level of the weather@home2 modelling system (w@h2: a successor to the well-known weather@home1 modelling system) which now produces an exceptionally large number of ensembles of simulations (>10,000). Having been designed for the investigation of the behavior of extreme weather under anthropogenic climate change, findings show that the performance of w@h2 in terms of climate variability may be more relevant than measures of the mean climatology. To some significant extent w@h2 model provides little, if any, predictive information for precipitation during the dry season, but may provide useful information during the monsoon seasons as well as skill to capture the Little Dry Season over the Guinea zone; predictive skills for the onset season suggest that the model is getting processes right. The w@h2 model is also able to reproduce all the annual characteristics of the surface maximum air temperature over the sub-region with skill to detect heat waves that usually ravage West Africa during the boreal spring. With synchronization > 80% the model has the ability to reliably / accurately simulate the actual anomaly signs of the observed climate parameters which is one of the special attributes of a model that is needed for seasonal climate predictions and applications. The large sample sizes produced by the w@h2 model are able to show that sampling quality of the tails of the distribution is no longer the primary constraint / source of uncertainty. The study further furnishes a prospective user with information on whether the model might be “useful or not” for a particular application.
... We choose the one that yields the highest sensitivity and is doable within the computational budget limit. We set the latter to be ≈ 50, 000 CPU core years which is roughly the computational power of Einstein@home (Anderson 2004;Anderson et al. 2006) running for one year. The optimal set-up uses a small fraction of the computing budget, has a coherence length ℎ = 1 yr, covers the entire parameter space with ≈ 2 · 10 16 templates and has of a total of 3 follow-up stages on independent data streams. ...
We consider stably rotating highly magnetised neutron stars and glitching pulsars. We discuss the prospects for detecting continuous gravitational waves from these sources below 20 Hz with next-generation ground-based facilities such as the Einstein Telescope and Cosmic Explorer and space-based observatories such as DECIGO and Big Bang Observer. We demonstrate that these constitute interesting science targets. We use a robust sensitivity estimation method for future searches based on demonstrated performance. We show that the spin-down upper limit on the gravitational wave amplitude of more than 90% of all highly magnetised pulsars and magnetars suitable for a years-long fully coherent search, exceeds the smallest gravitational wave amplitude estimated detectable with DECIGO and Big Bang Observer. We find that the hidden magnetar candidate PSR J1852+0040 can be detected by Cosmic Explorer if it is emitting at least at 20% of its spin-down luminosity. Finally, post-glitch transient continuous gravitational waves from magnetars are an interesting target for deci-Hz detectors, with all but one of the recorded glitches giving rise to a spin-down limit signal above the smallest detectable level.
... Distributed computing has proven to be a powerful approach for addressing computationally demanding problems, particularly in scientific fields 10,11 . Projects like Folding@home 12 , which leverages the collective power of volunteers to simulate protein folding, have shown the potential of distributed computing to tackle largescale biochemical problems. ...
This paper presents the smart distributed data factory (SDDF), an AI-driven distributed computing platform designed to address challenges in drug discovery by creating comprehensive datasets of molecular conformations and their properties. SDDF uses volunteer computing, leveraging the processing power of personal computers worldwide to accelerate quantum chemistry (DFT) calculations. To tackle the vast chemical space and limited high-quality data, SDDF employs an ensemble of machine learning (ML) models to predict molecular properties and selectively choose the most challenging data points for further DFT calculations. The platform also generates new molecular conformations using molecular dynamics with the forces derived from these models. SDDF makes several contributions: the volunteer computing platform for DFT calculations; an active learning framework for constructing a dataset of molecular conformations; a large public dataset of diverse ENAMINE molecules with calculated energies; an ensemble of ML models for accurate energy prediction. The energy dataset was generated to validate the SDDF approach of reducing the need for extensive calculations. With its strict scaffold split, the dataset can be used for training and benchmarking energy models. By combining active learning, distributed computing, and quantum chemistry, SDDF offers a scalable, cost-effective solution for developing accurate molecular models and ultimately accelerating drug discovery.
... The search algorithm was implemented in a way that allows for distributed computing. We are now adapting the algorithm to utilize additional computing power using the online distributing network BOINC (Anderson, 2004). Such large-scale computing efforts will be very promising in generating new, more elaborate formulas and interrelations. ...
Fundamental mathematical constants appear in nearly every field of science, from physics to biology. Formulas that connect different constants often bring great insight by hinting at connections between previously disparate fields. Discoveries of such relations, however, have remained scarce events, relying on sporadic strokes of creativity by human mathematicians. Recent developments of algorithms for automated conjecture generation have accelerated the discovery of formulas for specific constants. Yet, the discovery of connections between constants has not been addressed. In this paper, we present the first library dedicated to mathematical constants and their interrelations. This library can serve as a central repository of knowledge for scientists from different areas, and as a collaborative platform for development of new algorithms. The library is based on a new representation that we propose for organizing the formulas of mathematical constants: a hypergraph, with each node representing a constant and each edge representing a formula. Using this representation, we propose and demonstrate a systematic approach for automatically enriching this library using PSLQ, an integer relation algorithm based on QR decomposition and lattice construction. During its development and testing, our strategy led to the discovery of 75 previously unknown connections between constants, including a new formula for the `first continued fraction' constant , novel formulas for natural logarithms, and new formulas connecting and e. The latter formulas generalize a century-old relation between and e by Ramanujan, which until now was considered a singular formula and is now found to be part of a broader mathematical structure. The code supporting this library is a public, open-source API that can serve researchers in experimental mathematics and other fields of science.
... We thank Peter Brown for correspondence on bolide observations, and Bill Bottke and Robert Jedicke for discussions of results in earlier versions of this manuscript. Early versions of this research made use of the BOINC-based (Anderson 2004) orbit@home distributed computing network, and the participants who donated computing time on their personal computers are acknowledged. ...
Determining the size and orbital distribution of the population of near-Earth asteroids (NEAs) is the focus of intense research, with the most recent models converging to a population of approximately 1000 NEAs larger than 1 km and up to approximately NEAs with absolute magnitude . We present an analysis of the combined observations of nine of the leading asteroid surveys over the past two decades, and show that for an absolute magnitude , which is often taken as proxy for an average diameter larger than 1 km, the population of NEAs is , lower than other recent estimates. The population of small NEAs is estimated at for , and the number of decameter NEAs is lower than other recent estimates. This population tracks accurately the orbital distribution of recently discovered large NEAs, and produces an estimated Earth impact rate for small NEAs in good agreement with the bolide data.
... Resources may have different computational power and availability and may have different mechanisms to be accessed. Therefore, over the years, several tools have been created such as Condor [35], XtremWeb [36], BOINC [37], Nimrod [38], and OurGrid [39] to facilitate user access to these computational platforms. These tools mainly focus on managing users jobs looking into the computational infrastructure. ...
High Performance Computing (HPC) applications are essential for scientists and engineers to create and understand models and their properties. These professionals depend on the execution of large sets of computational jobs that explore combinations of parameter values. Avoiding the execution of unnecessary jobs brings not only speed to these experiments, but also reductions in infrastructure usage---particularly important due to the shift of these applications to HPC cloud platforms. Our hypothesis is that data generated by these experiments can help users in identifying such jobs. To address this hypothesis we need to understand the similarity levels among multiple experiments necessary for job elimination decisions and the steps required to automate this process. In this paper we present a study and a machine learning-based tool called JobPruner to support parameter exploration in HPC experiments. The tool was evaluated with three real-world use cases from different domains including seismic analysis and agronomy. We observed the tool reduced 93% of jobs in a single experiment, while improving quality in most scenarios. In addition, reduction in job executions was possible even considering past experiments with low correlations.
... In order to check the integrity of a MapReduce computation on a Desktop Grid [43,20], a replication based approach is suggested in [76], where reducers check results produced by mappers and the master process checks results produced by reducers. In addition, a MD5-based scheme is given to check outputs of mappers against a predefined digest code and Map function. ...
MapReduce is a programming system for distributed processing large-scale data in an efficient and fault tolerant manner on a private, public, or hybrid cloud. MapReduce is extensively used daily around the world as an efficient distributed computation tool for a large class of problems, e.g., search, clustering, log analysis, different types of join operations, matrix multiplication, pattern matching, and analysis of social networks. Security and privacy of data and MapReduce computations are essential concerns when a MapReduce computation is executed in public or hybrid clouds. In order to execute a MapReduce job in public and hybrid clouds, authentication of mappers-reducers, confidentiality of data-computations, integrity of data-computations, and correctness-freshness of the outputs are required. Satisfying these requirements shield the operation from several types of attacks on data and MapReduce computations. In this paper, we investigate and discuss security and privacy challenges and requirements, considering a variety of adversarial capabilities, and characteristics in the scope of MapReduce. We also provide a review of existing security and privacy protocols for MapReduce and discuss their overhead issues.
... In this paper, we introduce "Smart Data Factory," a framework that leverages active learning and volunteer-based distributed computing to construct a comprehensive, high-quality dataset of molecular conformations with their DFT-calculated properties. Volunteer computing has been effectively employed for scientific calculations in many projects before [10] [11], including for biochemical problems [12]. With the help of volunteer computing we hope to accelerate these DFT calculations by harnessing the collective processing power of numerous personal computers worldwide. ...
This paper presents the Smart Data Factory (SDF), an AI-driven distributed computing platform designed to address challenges in drug discovery by creating comprehensive datasets of molecular conformations and their properties. SDF uses volunteer computing, leveraging the processing power of personal computers worldwide to accelerate quantum chemistry (DFT) calculations. To tackle the vast chemical space and limited high-quality data, SDF employs an ensemble of machine learning models to predict molecular properties and selectively choose the most challenging data points for further DFT calculations. The platform also generates new molecular conformations using molecular dynamics with the forces derived from these models. SDF makes several contributions: the volunteer computing platform for DFT calculations; an active learning framework for constructing a dataset of molecular conformations; a large public dataset of diverse ENAMINE molecules with calculated energies; an ensemble of state-of-the-art ML models for accurate energy prediction. The energy dataset was generated to validate the SDF approach of reducing the need for extensive calculations. With its strict scaffold split, the dataset can be used for training and benchmarking energy models. By combining active learning, distributed computing, and quantum chemistry, SDF offers a scalable, cost-effective solution for developing accurate molecular models and ultimately accelerating drug discovery.
... The simulation experiments are run within CPDN 10,11 using the Berkeley Open Infrastructure for Network Computing (BOINC) 16 framework to distribute a large number of individual computational tasks. This system utilises the computational power of publicly volunteered computers. ...
Large ensembles of global temperature are provided for three climate scenarios: historical (2006–16), 1.5 °C and 2.0 °C above pre-industrial levels. Each scenario has 700 members (70 simulations per year for ten years) of 6-hourly mean temperatures at a resolution of 0.833° ´ 0.556° (longitude ´ latitude) over the land surface. The data was generated using the climateprediction.net (CPDN) climate simulation environment, to run HadAM4 Atmosphere-only General Circulation Model (AGCM) from the UK Met Office Hadley Centre. Biases in simulated temperature were identified and corrected using quantile mapping with reference temperature data from ERA5. The data is stored within the UK Natural and Environmental Research Council Centre for Environmental Data Analysis repository as NetCDF V4 files.
... BOINC [1][2] is a middleware system for Volunteer Computing(VC). To run applications of multiple discipline on volunteer computers, BOINC provides a framework and components for VC projects, and these components should be customized based on requirements of target applications. ...
Delphes is a C++ framework to perform a fast multipurpose detector response simulation. The Circular Electron Positron Collider (CEPC) experiment runs fast simulation with a modified Delphes based on its own scientific objectives. The CEPC fast simulation with Delphes is a High Throughput Computing (HTC) application with small input and output files. Besides, to compile and run Delphes, only ROOT software is necessary.Therefore, all these features make it appropriate to run CEPC fast simulation as a Volunteer Computing application. As a result, a BOINC project named HEP@home is developed to run fast simulation with Delphes for CEPC. This paper describes the internal structure of the project, pre and post data operations, and its development status.
... This section explores the paradigm of cloud computing, focusing on its service models (IaaS, PaaS, SaaS) and deployment models (public, private, hybrid). The research elucidates the benefits of cloud computing in terms of resource scalability, cost-effectiveness, and accessibility [15]. Case studies demonstrate how organizations leverage cloud services for data storage, computation, and scalable deployment of applications. ...
This research paper explores the landscape of multiple computation paradigms, seeking to understand their synergies, applications, and potential for advancing computational capabilities. The study provides a comprehensive overview of various models, methodologies, and technologies associated with multiple computation, including parallel computing, distributed computing, and cloud computing. Through in-depth analysis and case studies, this research aims to elucidate the strengths, challenges, and future directions in harnessing the power of multiple computation for diverse domains.
... • This scheme is sent to a server, where it is parsed into smaller chunks, designed to compute in reasonable time on a standard home computer. The distribution of the computing chunks is done using the Berkeley Open Infrastructure for Network Computing (BOINC) [16]. ...
In recent decades, a growing number of discoveries in fields of mathematics have been assisted by computer algorithms, primarily for exploring large parameter spaces that humans would take too long to investigate. As computers and algorithms become more powerful, an intriguing possibility arises - the interplay between human intuition and computer algorithms can lead to discoveries of novel mathematical concepts that would otherwise remain elusive. To realize this perspective, we have developed a massively parallel computer algorithm that discovers an unprecedented number of continued fraction formulas for fundamental mathematical constants. The sheer number of formulas discovered by the algorithm unveils a novel mathematical structure that we call the conservative matrix field. Such matrix fields (1) unify thousands of existing formulas, (2) generate infinitely many new formulas, and most importantly, (3) lead to unexpected relations between different mathematical constants, including multiple integer values of the Riemann zeta function. Conservative matrix fields also enable new mathematical proofs of irrationality. In particular, we can use them to generalize the celebrated proof by Ap\'ery for the irrationality of . Utilizing thousands of personal computers worldwide, our computer-supported research strategy demonstrates the power of experimental mathematics, highlighting the prospects of large-scale computational approaches to tackle longstanding open problems and discover unexpected connections across diverse fields of science.
... To accelerate the process of manual annotation, research teams are increasingly adopting novel approaches, such as working with 'citizen scientists' to label large datasets. 'Citizen science' is a term that encompasses a diverse and growing set of research practices, from donating idle computer time to research projects (Anderson 2004), self-reporting the symptoms of illnesses, such as coronavirus disease (COVID) (Varsavsky et al. 2021), and working in community biology labs (Landrain et al. 2013), to analysing protein structures online (Cooper et al. 2010). Yet, despite this diversity, citizen science may be understood simply as involvement of (presumed) non-professionals in research (Bonney 1996). ...
Public participation in research, also known as citizen science, is being increasingly adopted for the analysis of biological volumetric data. Researchers working in this domain are applying online citizen science as a scalable distributed data analysis approach, with recent research demonstrating that non-experts can productively contribute to tasks such as the segmentation of organelles in volume electron microscopy data. This, alongside the growing challenge to rapidly process the large amounts of biological volumetric data now routinely produced, means there is increasing interest within the research community to apply online citizen science for the analysis of data in this context. Here, we synthesise core methodological principles and practices for applying citizen science for analysis of biological volumetric data. We collate and share the knowledge and experience of multiple research teams who have applied online citizen science for the analysis of volumetric biological data using the Zooniverse platform (www.zooniverse.org). We hope this provides inspiration and practical guidance regarding how contributor effort via online citizen science may be usefully applied in this domain.
... Already in 2010 the BOINC (Berkeley Open Infrastructure for Network Computing) system (Anderson, 2004) was used by the Help Conquer Cancer (HCC) project for automatic scoring of images from high-throughput protein crystallization trials (Cumbaa & Jurisica, 2010;Kotseruba et al., 2012). Public resource or volunteer computing uses internet-connected computers whose owners voluntarily share unused capacities with scientists having a huge computational demand but low budget. ...
To avoid the time-consuming and often monotonous task of manual inspection of crystallization plates, a Python-based program to automatically detect crystals in crystallization wells employing deep learning techniques was developed. The program uses manually scored crystallization trials deposited in a database of an in-house crystallization robot as a training set. Since the success rate of such a system is able to catch up with manual inspection by trained persons, it will become an important tool for crystallographers working on biological samples. Four network architectures were compared and the SqueezeNet architecture performed best. In detecting crystals AlexNet accomplished a better result, but with a lower threshold the mean value for crystal detection was improved for SqueezeNet. Two assumptions were made about the imaging rate. With these two extremes it was found that an image processing rate of at least two times, but up to 58 times in the worst case, would be needed to reach the maximum imaging rate according to the deep learning network architecture employed for real-time classification. To avoid high workloads for the control computer of the CrystalMation system, the computing is distributed over several workstations, participating voluntarily, by the grid programming system from the Berkeley Open Infrastructure for Network Computing (BOINC). The outcome of the program is redistributed into the database as automatic real-time scores (ARTscore). These are immediately visible as colored frames around each crystallization well image of the inspection program. In addition, regions of droplets with the highest scoring probability found by the system are also available as images.
... Кожен комп'ютер виконує свою частину роботи, після чого результати об'єднуються в центральному сервері. Користувачі можуть долучатися до різних наукових проєктів, які потребують великої обчислювальної потужності, та внести свій внесок у розв'язання важливих наукових проблем [1]. ...
У роботі проаналізована робота проєкту BOINC, платформи для наукових досліджень, яка дозволяє залучати волонтерів з усього світу та використовувати вільні ресурси особистих комп'ютерів для проведення складних наукових досліджень. Завдяки BOINC зменшується навантаження на наукові центри та скорочується час, необхідний для проведення досліджень у різних сферах, включаючи медицину, астрономію, фінанси, енергетику та транспорт.
... To accelerate the process of manual annotation, research teams are increasingly adopting novel approaches, such as working with 'citizen scientists' to label large datasets. 'Citizen science' is a term that encompasses a diverse and growing set of research practices, from donating idle computer time to research projects (Anderson, 2004), self-reporting the symptoms of illnesses such as COVID (Varsavsky et al, 2021), and working in community biology labs (Landrain et al, 2013), to analysing protein structures online (Cooper et al, 2010). Yet, despite this diversity, citizen science may be understood simply as involvement of (presumed) non-professionals in research (Bonney, 1996). ...
Public participation in research, also known as citizen science, is being increasingly adopted for the analysis of biological volumetric data. Researchers working in this domain are applying online citizen science as a scalable distributed data analysis approach, with recent research demonstrating that non-experts can productively contribute to tasks such as the segmentation of organelles in volume electron microscopy data. This, alongside the growing challenge to rapidly process the large amounts of biological volumetric data now routinely produced, means there is increasing interest within the research community to apply online citizen science for the analysis of data in this context. Here, we synthesise core methodological principles and practices for applying citizen science for analysis of biological volumetric data. We collate and share the knowledge and experience of multiple research teams who have applied online citizen science for the analysis of volumetric biological data using the Zooniverse platform (www.zooniverse.org). We hope this provides inspiration and practical guidance regarding how contributor effort via online citizen science may be usefully applied in this domain.
The need for computational resources grows as computational algorithms gain popularity in different sectors of the scientific community. Sequential codes need to be converted to parallel versions to optimize the use of these resources. Maintaining a local infrastructure for the execution of distributed computing, through desktop grids, for example, has been replaced in favor of cloud platforms that abstract the complexity of these local infrastructures. Unfortunately, the cost of accessing these resources could leave out various studies that could be carried by a simpler infrastructure. In this article, we present a platform for distributing computer simulations on resources available on a local network using container virtualization that abstracts the complexity needed to configure these execution environments and allows any user can benefit from this infrastructure. Simulations could be developed in any programming language (such as Python, Java, C, and R) and with specific execution needs within reach of the scientific community in a general way. We will present results obtained in running simulations that required more than 1000 runs with different initial parameters and various other experiments that benefited from using the platform.
Weather and climate forecasting, using climate models, have become essential tools and life-savers in the West African region; in spite of the fact that climate models do not fully comply with attributes of forecast qualities—RASAP: reliability, association, skill, accuracy, and precision. The objective of this paper is to quantitatively evaluate, in comparison to CRU and ERA5 datasets, the RASAP compliance-level of the weather@home2 modeling system (w@h2). Findings from some statistical evaluations show that, to a moderately significant extent, w@h2 model provides useful information during the monsoon seasons; skills to capture the Little Dry Season over the Guinea zone; predictive skills for the onset season; ability to reproduce all the annual characteristics of the surface maximum air temperature over the region; as well as skill to detect heat waves that usually ravage West Africa during the boreal spring. The model displays traces of attributes that are needed for seasonal climate predictions and applications. Deficiencies in the quantitative reproducibility point to the facts that the model does provide a reliability akin to that of regional climate models. This paper further furnishes a prospective user with information on whether the model might be “useful or not” for a particular application.
We conduct two searches for continuous, nearly monochromatic gravitational waves originating from the central compact objects in the supernova remnants Cassiopeia A and Vela Jr. using public LIGO data. The search for Cassiopeia A targets signal frequencies between 20 Hz and 400 Hz; the Vela Jr. search between 400 Hz and 1700 Hz, and both investigate the broadest set of waveforms ever considered with highly sensitive deterministic search methods. Above 1500 Hz the Vela Jr. search is the most sensitive carried out thus far, improving on previous results by over 300\%. Above 976 Hz these results improve on existing ones by 50\%. In all we investigate over waveforms, leveraging the computational power donated by thousands of Einstein@Home volunteers. We perform a 4-stage follow-up on more than 6 million waveforms. None of the considered waveforms survives the follow-up scrutiny, indicating no significate detection candidate. Our null results constrain the maximum amplitude of continuous signals as a function of signal frequency from the targets. The most stringent 90\% confidence upper limit for Cas A is near 200 Hz, and for Vela Jr. it is near 400 Hz. Translated into upper limits on the ellipticity and r-mode amplitude, our results probe physically interesting regions: for example the ellipticity of Vela Jr. is constrained to be smaller than across the frequency band, with a tighter constraint of less than at the highest frequencies.
The ATLAS experiment has developed extensive software and distributed computing systems for Run 3 of the LHC. These systems are described in detail, including software infrastructure and workflows, distributed data and workload management, database infrastructure, and validation. The use of these systems to prepare the data for physics analysis and assess its quality are described, along with the software tools used for data analysis itself. An outlook for the development of these projects towards Run 4 is also provided.
We perform a search for continuous nearly monochromatic gravitational waves from the central compact objects associated with the supernova remnants Vela Jr. and G347.3. Over 10 ¹⁸ different waveforms are considered, covering signal frequencies between 20 and 1300 Hz (20 and 400 Hz) for G347.3-0.5 (Vela Jr.) and a very broad range of frequency derivatives. The data set used for this first search is from the second observing run of LIGO (O2). Thousands of volunteers donating compute cycles through the computing project Einstein@Home have made this endeavor possible. Following the Einstein@Home search, we perform multistage follow-ups of over 5 million waveforms. The threshold for selecting candidates from the Einstein@Home search is such that, after the multistage follow-up, we do not expect any surviving candidate due to noise. The very last stage uses a different data set, namely, the LIGO O3 data. We find no significant signal candidate for either targets. Based on this null result, for G347.3-0.5, we set the most constraining upper limits to date on the amplitude of gravitational-wave signals, corresponding to deformations below 10 ⁻⁶ in a large part of the search band. At the frequency of best strain sensitivity, near 161 Hz, we set 90% confidence upper limits on the gravitational-wave intrinsic amplitude of h 0 90 % ≈ 6.2 × 10 − 26 . Over most of the frequency range, our upper limits are a factor of 10 smaller than the indirect age-based upper limit. For Vela Jr., near 163 Hz, we set h 0 90 % ≈ 6.4 × 10 − 26 . Over most of the frequency range, our upper limits are a factor of 15 smaller than the indirect age-based upper limit. The Vela Jr. upper limits presented here are slightly less constraining than the most recent upper limits of R. Abbott et al., but they apply to a broader set of signals.
Recently, various emerging technologies have been introduced to host IoT applications. Edge computing, utilizing volunteer devices, could be a feasible solution due to the significant and underutilized resources at the edge. However, cloud providers are still reluctant to offer it as an edge infrastructure service because of the unpredictable nature of volunteer resources. Volunteer edge computing introduces challenges such as reliability, trust, and availability. Testing this infrastructure is prohibitively expensive and not feasible in real-world scenarios. This emerging technology will not be fully realized until dedicated research and development efforts have substantiated its potential for running reliable services. Therefore, this paper proposes VonEdgeSim, a simulation of volunteer edge computing. To the best of our knowledge, it is the first and only simulation capable of mimicking volunteer behavior at the edge. Researchers and developers can utilize this simulation to test and develop resource management models. We conduct experiments with various IoT applications, including Augmented Reality, Infotainment, and Health Monitoring. Our results show that incorporating volunteer devices at the edge can significantly enhance system performance by reducing total task delay, and improving task execution time. This emphasizes the potential of volunteers to provide reliable services in an edge computing environment. The simulation code is publicly available for further development and testing.
The widespread adoption of utility-based real-time applications has placed the necessity of widescale deployment of edge computing infrastructure. Crowdsourced edge computing is deemed a suitable way out. On the other hand, a collection of today’s powerful smart mobile devices (SMDs) can cumulatively offer high-performance computing. The public-owned SMDs are utilized opportunistically to form a dynamic ad-hoc computing grid known as mobile crowd computing (MCC). This paper aspires to establish a proof-of-concept for the feasibility and use of MCC as a sustainable edge computing solution (MCC-edge). A typical smart HVAC system of an office building has been considered for the experiment case. We aim to process the HVAC data in real-time using the MCC-edge setup within the building for auto adjustment of the AC controller and error notifications. To maintain the ideal comfort level of the occupants, we present an extensive calculation using the dew point and heat index of the room. A high-level layered architecture of the MCC-edge for HVAC is presented along with a general framework of the MCC-edge. We report the module-wise design and implementation procedures with exhaustive details. The performance of MCC-edge is statistically compared with the commercial edge and cloud computing solutions in terms of cost, energy consumption, latency, and environmental impact, showing a significant advantage over the two. Every procedural detail of each module's design, development, and implementation is meticulously presented, which would aid interested readers and researchers in rebuilding such an application.
In recent decades, a growing number of discoveries in mathematics have been assisted by computer algorithms, primarily for exploring large parameter spaces. As computers become more powerful, an intriguing possibility arises—the interplay between human intuition and computer algorithms can lead to discoveries of mathematical structures that would otherwise remain elusive. Here, we demonstrate computer-assisted discovery of a previously unknown mathematical structure, the conservative matrix field. In the spirit of the Ramanujan Machine project, we developed a massively parallel computer algorithm that found a large number of formulas, in the form of continued fractions, for numerous mathematical constants. The patterns arising from those formulas enabled the construction of the first conservative matrix fields and revealed their overarching properties. Conservative matrix fields unveil unexpected relations between different mathematical constants, such as π and ln(2), or e and the Gompertz constant. The importance of these matrix fields is further realized by their ability to connect formulas that do not have any apparent relation, thus unifying hundreds of existing formulas and generating infinitely many new formulas. We exemplify these implications on values of the Riemann zeta function ζ (n), studied for centuries across mathematics and physics. Matrix fields also enable new mathematical proofs of irrationality. For example, we use them to generalize the celebrated proof by Apéry of the irrationality of ζ (3). Utilizing thousands of personal computers worldwide, our research strategy demonstrates the power of large-scale computational approaches to tackle longstanding open problems and discover unexpected connections across diverse fields of science.
Blockchain infrastructures have emerged as a disruptive technology and have led to the realization of cryptocurrencies (peer‐to‐peer payment systems) and smart contracts. They can have a wide range of application areas in e‐Science due to their open, public nature and global accessability in a trustless manner. We propose and implement a smart contract called eBlocBroker, which is an autonomous blockchain‐based middleware system for volunteer computing and providing data resources for e‐Science. The eBlocBroker infrastructure connects requesters who need to combine applications (jobs) with datasets and run them via an Ethereum‐based private blockchain network (Bloxberg) on providers that utilize computational and data resources on clouds or home servers. It uses cloud storage, such as B2DROP, IPFS, or Google Drive, to store and transfer data between requesters and providers. Each provider utilizes the Slurm workload manager to execute jobs submitted through eBlocBroker. In this paper, we demonstrate how an autonomous organization programmed as a smart contract can be used to deploy a marketplace that supports data and computation‐intensive research projects. We propose a cost model implemented as a function in the smart contract which calculates and records computation and dataset usage costs. We develop a Python‐based system to communicate with eBlocBroker and orchestrate jobs' execution on the provider's end. We present eBlocBroker's features, infrastructure, implementation, algorithms and experimental results.
The paper is dedicated to preliminary results and validation of the distributed solution for causal role attribution in gene network expansion problem. The key ingredients of the solution are the web application based on Shiny framework, RBOINC backend as an interface between R language and BOINC desktop grid framework, and parallel (multicore) implementation of the Peter-Clark (PC) algorithm for causal role attribution. The approach is technically validated on a gene network expansion problem for the grapevine (Vitis vinifera).
KeywordsCausal Relationship DiscoveryDistributed ComputingRBOINCGene Network Expansion
Limiting global mean temperature rise to 1.5 °C is increasingly out of reach. Here we show the impact on global cooling demand in moving from 1.5 °C to 2.0 °C of global warming. African countries have the highest increase in cooling requirements. Switzerland, the United Kingdom and Norway (traditionally unprepared for heat) will suffer the largest relative cooling demand surges. Immediate and unprecedented adaptation interventions are required worldwide to be prepared for a hotter world.
We performed a billion locality sensitive hash comparisons between artificially generated data samples to answer the critical question - can we verify the "correctness" of generative AI output in a non-deterministic, trustless, decentralized network? We generate millions of data samples from a variety of open source diffusion and large language models and describe the procedures and trade-offs between generating more verses less deterministic output in a heterogenous, stochastic network. Further, we analyze the outputs to provide empirical evidence of different parameterizations of tolerance and error bounds for verification. Finally, given that we have the generated an enormous amount of simulated data, we also release a new training dataset called ImageNet-Gen for use in augmenting existing training pipelines. For our results, we show that with a majority vote between three independent verifiers, we can detect image generated perceptual collisions in generated AI with over 99.89% probability and less than 0.0267% chance of intra-class collision. For large language models (LLMs), we are able to gain 100% consensus using greedy methods or n-way beam searches to generate consensus demonstrated on different LLMs. In the context of generative AI training, we pinpoint and minimize the major sources of stochasticity and present gossip and synchronization training techniques for verifiability. Thus, this work provides a practical, solid foundation for AI verification and consensus for the minimization of trust in a decentralized network.
The effects of environmental pollution and global warming have become a reality and severe. In addition to other causes, wide adoption and huge demands for computational resources have aggravated it significantly. The production process of the computing devices involves hazardous and toxic substances which not only harm human and other living beings’ health but also contaminate the water and soil. The production and operations of these computers on a large scale also result in massive energy consumption and greenhouse gas generation. Moreover, the low use cycle of these devices produces a huge amount of not easy-to-decompose e-waste. In this outlook, instead of buying new devices, it is advisable to use the existing resources to their fullest, which will minimize the environmental penalties of production and e-waste. In this study, we advocate for mobile crowd computing (MCC) to ease off the use of centralized high-performance computers (HPCs) such as data centres and supercomputers by utilising SMDs (smart mobile devices) as computing devices. We envision establishing MCC as the most feasible computing system solution for sustainable computing. Successful real-world implementation of MCC involves several challenges. In this study, we primarily focus on resource management. We devised a methodological and structured approach for resource profiling. We present the resource selection problem as an MCDM (multi-criteria decision making) problem and compared five MCDM approaches of dissimilar families to find the appropriate methodology for dynamic resource selection in MCC. To improve the overall performance of MCC, we present two scheduling algorithms, considering different objectives such as makespan, resource utilisation, load balance, and energy efficiency. We propose a deep learning based resource availability prediction to minimise the job offloading in a local MCC. We further propose a mobility-aware resource provisioning scheme for a P2P MCC. Finally, we present a proof-of-concept of the proposed MCC model as a working prototype. For this, we consider a smart HVAC (heating, ventilation, and cooling) system in a multistorey building and use MCC as a local edge computing infrastructure. The successful implementation of the prototype proves the feasibility and potential of MCC as alternative sustainable computing.
It has been reported [25] that life holds but two certainties, death and taxes. And indeed, it does appear that any society, and in the context of this article, any large-scale distributed system, must address both death (failure) and the establishment and maintenance of infrastructure (which we assert is a major motivation for taxes, so as to justify our title!). Two supposedly new approaches to distributed computing have emerged in the past few years, both claiming to address the problem of organizing large-scale computational societies: peer-to-peer (P2P) [15, 36, 49] and Grid computing [21]. Both approaches have seen rapid evolution, widespread deployment, successful application, considerable hype, and a certain amount of (sometimes warranted) criticism. The two technologies appear to have the same final objective, the pooling and coordinated use of large sets of distributed resources, but are based in different communities and, at least in their current designs, focus on different requirements.
Search for Extraterrestrial Intelligence (SETI) is a scientific area whose goal is to detect intelligent life outside Earth. One approach, known as radio SETI, uses radio telescopes to listen for narrow-bandwidth radio signals from space. Such signals are not known to occur naturally, so a detection would provide evidence of extraterrestrial technology. SETI@home uses millions of computers in homes and offices around the world to analyze radio signals from space. This article describes the design and implementation of SETI@home and discusses its relevance to future distributed systems. It is introduced as a new step in experimenting public-resource computing. To attract and keep users, a project must explain and justify its goals, providing compelling views of local and global progress. Screensaver graphics are an excellent medium for displays that also provide a form of viral marketing. Moreover, the success of public-resource computing projects has the ancillary benefit of increasing public awareness of science and democratizing, to an extent, the allocation of research resources.
Global computing achieves high throughput computing by harvesting
a very large number of unused computing resources connected to the
Internet. This parallel computing model targets a parallel architecture
defined by a very high number of nodes, poor communication performance
and continuously varying resources. The unprecedented scale of the
global computing architecture paradigm requires us to revisit many basic
issues related to parallel architecture programming models, performance
models, and class of applications or algorithms suitable for this
architecture. XtremWeb is an experimental global computing platform
dedicated to provide a tool for such studies. The paper presents the
design of XtremWeb. Two essential features of this design are
multi-applications and high-performance. Accepting multiple applications
allows institutions or enterprises to set up their own global computing
applications or experiments. High-performance is ensured by scalability,
fault tolerance, efficient scheduling and a large base of volunteer PCs.
We also present an implementation of the first global application
running on XtremWeb
OceanStore is a utility infrastructure designed to span the globe and provide continuous access to persistent information. Since this infrastructure is comprised of untrusted servers, data is protected through redundancy and cryptographic techniques. To improve performance, data is allowed to be cached anywhere, anytime. Additionally, monitoring of usage patterns allows adaptation to regional outages and denial of service attacks; monitoring also enhances performance through pro-active movement of data. A prototype implementation is currently under development.
Global Computing harvest the idle time of Internet connected computers to run very large distributed applications. The unprecedented scale of the GCS paradigm requires to revisit the basic issues of distributed systems: performance models, security, fault-tolerance and scalability. The first parts of this paper review recent work in Global Computing, with particular interest in Peer-to-Peer systems. In the last section, we present XtremWeb, the Global Computing System we are currently developing.
Global Computing is a particular modality of Grid Computing targeting massive parallelism, Internet computing and cycle-stealing. This new computing infrastructure has been shown to be exposed to a new type of attacks, where authentication is not relevant, network security techniques are not sufficient, and result-checking algorithms may be unavailable. The behavior of a Global Computing System, which is likely to be bimodal, nevertheless offers an opportunity for a probabilistic verification process that is efficient in the most frequent cases, and degrades gracefully as the problem becomes more difficult. For the two cases of a system based on anonymous volunteers, and a better controlled system, we propose probabilistic tests which self-adapt to the behavior of the computing entities.
For buyers and sellers alike, there's no better way to earn one another's trust in online interactions.
The exploitation of idle cycles on pervasive desktop PC systems offers the opportunity to increase the available computing power by orders of magnitude (10x - 1000x). However, for desktop PC distributed computing to be widely accepted within the enterprise, the systems must achieve high levels of efficiency, robustness, security, scalability, manageability, unobtrusiveness, and openness/ease of application integration. We describe the Entropia distributed computing system as a case study, detailing its internal architecture and philosophy in attacking these key problems. Key aspects of the Entropia system include the use of: 1) binary sandboxing technology for security and unobtrusiveness, 2) a layered architecture for efficiency, robustness, scalability and manageability, and 3) an open integration model to allow applications from many sources to be incorporated. Typical applications for the Entropia System includes molecular docking, sequence analysis, chemical structure modeling, and risk management. The applications come from a diverse set of domains including virtual screening for drug discovery, genomics for drug targeting, material property prediction, and portfolio management. In all cases, these applications scale to many thousands of nodes and have no dependences between tasks. We present representative performance results from several applications that illustrate the high performance, linear scaling, and overall capability presented by the Entropia system.
Atomistic simulations of protein folding have the potential to be a great complement to experimental studies, but have been severely limited by the time scales accessible with current computer hardware and algorithms. By employing a worldwide distributed computing network of tens of thousands of PCs and algorithms designed to efficiently utilize this new many-processor, highly heterogeneous, loosely coupled distributed computing paradigm, we have been able to simulate hundreds of microseconds of atomistic molecular dynamics. This has allowed us to directly simulate the folding mechanism and to accurately predict the folding rate of several fast-folding proteins and polymers, including a nonbiological helix, polypeptide alpha-helices, a beta-hairpin, and a three-helix bundle protein from the villin headpiece. Our results demonstrate that one can reach the time scales needed to simulate fast folding using distributed computing, and that potential sets used to describe interatomic interactions are sufficiently accurate to reach the folded state with experimentally validated rates, at least for small proteins.
Increasingly, computing addresses collaboration, data sharing, and interaction modes that involve distributed resources, resulting in an increased focus on the interconnection of systems both within and across enterprises. These evolutionary pressures have led to the development of Grid technologies. The authors' work focuses on the nature of the services that respond to protocol messages. Grid provides an extensible set of services that can be aggregated in various ways to meet the needs of virtual organizations, which themselves can be defined in part by the services they operate and share
The BitTorrent file distribution system uses tit-for-tat as a method of seeking pareto efficiency. It achieves a higher level of robustness and resource utilization than any currently known cooperative technique. We explain what BitTorrent does, and how economic methods are used to achieve that goal.
This paper presents and evaluates the storage management and caching in PAST, a large-scale peer-to-peer persistent storage utility. PAST is based on a self-organizing, Internet-based overlay network of storage nodes that cooperatively route file queries, store multiple replicas of files, and cache additional copies of popular files. In the PAST system, storage nodes and files are each assigned uniformly distributed identifiers, and replicas of a file are stored at nodes whose identifier matches most closely the file's identifier. This statistical assignment of files to storage nodes approximately balances the number of files stored on each node. However, non-uniform storage node capacities and file sizes require more explicit storage load balancing to permit graceful behavior under high global storage utilization; likewise, non-uniform popularity of files requires caching to minimize fetch distance and to balance the query load. We present and evaluate PAST, with an emphasis on its storage management and caching system. Extensive trace-driven experiments show that the system minimizes fetch distance, that it balances the query load for popular files, and that it displays graceful degradation of performance as the global storage utilization increases beyond 95%.
Emerging high-performance applications require the ability to exploit diverse, geographically distributed resources. These applications use high-speed networks to integrate supercomputers, large databases, archival storage devices, advanced visualization devices, and/or scientific instruments to form networked virtual supercomputers or metacomputers. While the physical infrastructure to build such systems is becoming widespread, the heterogeneous and dynamic nature of the metacomputing environment poses new challenges for developers of system software, parallel tools, and applications. In this article, we introduce Globus, a system that we are developing to address these challenges. The Globus system is intended to achieve a vertically integrated treatment of application, middleware, and network. A low-level toolkit provides basic mechanisms such as communication, authentication, network information, and data access. These mechanisms are used to construct various higher-leve...
Incentives Build Robustness in BitTorrent Workshop on Economics of P2P systems
- B Cohen
B. Cohen. " Incentives Build Robustness in BitTorrent ", Workshop on Economics of P2P systems. June 2003.
Oceanstore: An architecture for global-scale persistent storage
- J Kubiatowicz
- D Bindel
- Y Chen
- P Eaton
- D Geels
- R Gummadi
- S Rhea
- H Weatherspoon
- W Weimer
- C Wells
- B Zhao