Dataset - Science topic

Chukwudi Samuel Ekwezuo Hi

The JRA-55 data I have is actually in GRIB format (most probably). The file has no extension, but the JRA 55 documentation mentions that the data is in GRIB format.

BU-3DFE

According to No Free Lunch Theorem, there is no dominant algorithm in all dataset. Thus, making the code available in research paper with reproducible capability for several other research and other researcher point-of-view will increase the number of citation of the original paper and the rank of the journal

Thank you all very much.

Ajay Krishan Gairola Are you getting slower performance when you train the model or when you load the data? Perhaps you should use Tensorflow 2.0 data pipeline which is specifically optimized for such scenarios.

Computer Vision. Discovery Radiomics. Evolutionary Deep Intelligence. Image Segmentation/Classification

GOOD LUCK

Hi,

you need to compare the disease versus control samples when doing the statistical test. I didn't fully get if the problem is that you lack the information of which samples belongs to which category or it is a coding problem. As for the former, usually datasets have a metadata file in which the sample names you find in the gene expression table are present, and the treatment information is included. If it is a coding problem, you can index the sample names to divide the data into Asthmatic, Healthy and COPD.

David Schmidt Thanks a lot :)

Marcus W. Beck Yes, I am still on the hunt for more data! Thank you for sharing the links to the Tampa Bay monitoring data, I did not know about them. I will take a look at them!

Dear Chen Yijia ,

The application of deep learning in the field of drug discovery brings the development and expansion of molecular generative models along with new challenges in this field. One of challenges in de novo molecular generation is how to produce new reasonable molecules with desired pharmacological, physical, and chemical properties.

Regards,

Shafagat

The following python code could Average Multiple Curves and plot them.

I've shared it on Github and it's available from the following link:

Hi, you can use a dataset in Colab by using Google Drive.

Why date 2 to 30 or 31 is deactivate? how get the information of lead hour forecast of different dates of the month?

@Shafagat

Dear Shafagat,

Thank you for your reply. I am wondering do you know any more recent data? Something like this but not station data.

Best regards,

Haochen

I'm not sure if I understood correctly but I would use mathematica. One can simply make an interpolation for f1(x), f2(x) and f3(x):

(*Data to be fitted,coeffients like 0.005,0.12 etc to be found from the fitting*)

fxData=Table[{x,RandomReal[{0.997,1.003}] (0.005 x+0.12 Sqrt[x]+0.18 Exp[-x]-0.015 x^1.2)},{x,0,10,0.01}];

(*other data,for fxData,f1xData and so on one can use Import[],here are just random lists as an example*)

f1xData=Table[{x,Sqrt[x]},{x,0,10,0.01}];

f2xData=Table[{x,Exp[-x]},{x,0,10,0.01}];

f3xData=Table[{x,x^1.2},{x,0,10,0.01}];

(*Interpolation of f1x,f2x,f3x*)

f1x=Interpolation[f1xData];

f2x=Interpolation[f2xData];

f3x=Interpolation[f3xData];

(*fitting function*)

g[x_,a_,b_,c_,d_]:=a x+b f1x[x]+c f2x[x]-d f3x[x]

(*fitting*)

nlm=NonlinearModelFit[fxData,g[x,a,b,c,d],{a,b,c,d},x];

(*plot and parameter table*)

Show[{ListPlot[fxData,PlotRange->All],Plot[nlm[x],{x,0,10},PlotStyle->Red]}]

nlm["ParameterTable"]

Copy and paste should work :)

Check with the Georgia Tech Behavioral Medicine department. I know they were doing studies on individuals' gaits.

The imbalanced data is a common problem especially in Medicine.

I suggest you need to perform the following:

1- Split your entire data into train and test sets. The ratio is usually 80% for train set and 20% for test set. Make sure you stratify your split meaning that your train and test should have the same imbalanced ratio as your original data. As an example you can use sklearn library which does split your data and also perform stratify as below:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, shuffle=True, stratify=y)

2- Use up-sampling or down-sampling techniques to balance your data ONLY on your train set. DO NOT perform this on your test sets! Your test should remain intact!

3- Fit your ML model on your train set which is already balanced from the previous step.

4- Evaluate your model accuracy on your test set which is already imbalanced!

Hello Jan,

If your goal was to identify some "optimal" subset of IVs to best explain differences on an outcome of importance to you, stepwise models are among the least dependable means to accomplish this goal. In point of fact, you are in no way guaranteed to identify the best ensemble of IVs (for a given proportion of the total available IVs) via step methods. As well, they suffer from a number of other technical issues.

There are far better options available. Have a look at this list: https://scikit-learn.org/stable/modules/linear_model.html (and note the adaptive lasso option, among others).

Finally, with respect to your question: Why do the relative sizes of regression coefficients change as you add IVs to a model?

Because of overlap/redundancy among IVs (when excessive, this is referred to as multicollinearity). Later-entered variables can supplant earlier-entered variables, even though they might not have been as potent an individual IV as the first IV entered. If all IVs were genuinely independent of each other, then sequence of entry into a model would not change the explanatory power of a given IV as the model grew in number of predictors.

Good luck with your work.

To randomly select rows (packets) by using Pandas DataFrame:

a)

df = df.sample() // Randomly select a single row

b)

df = df.sample(n=500) // Randomly select a specified number of rows. For example, to select 500 random rows, set n=500

That can hardly be said in such abstract terms. It depends on how good the public data set is and how good your own is. The best thing is to use both and compare the results.

1. F0 would be one indicator for gender influence on voice.

2. many parameters have been evaluated to answer this question.

3. By acoustic analysis, i assume you mean pitch and intensity analysis. the parameters may differ depending on analysis technique one uses.

4. commonly age wise values are established as well as gender wise values are established for F0, harmonic to noise ratio, Jitter, Shimmer to name few. we use such norms routinely in clinics while trying to identifying abnormalities in voice.

5. cepstral peak- CPP and related parameters are often used in cepstral analysis

6. routine search on pubmed should give you lots of articles on this.

No!

If you're using the mean of multiple Likert-scale items, you're already treating the data as continuous (i.e., interval), in which case you can go ahead with Pearson correlation.

An alternative option is to use the median or mode of the Likert-scale items instead and conduct an ordinal regression.

Dear Sakib,

There are literally thousands of freely available data sets worldwide. What exactly do you need? No instrument is perfect and each data set has its own advantages and drawbacks. You should select those that are most appropriate for your purposes, and in particular determine your accuracy requirements, as they will imply close looks at the calibration issues as well as considerations regarding post-processing. Once you have clearly identified the parameters you need, the spatial and temporal extents and resolutions required, and the minimum accuracy needed, then you can search for the best inputs for your purpose.

By the way, NASA does offer a wide range of data sets but it is not the only source of information: the European Space Agency (ESA), as well as national space agencies of Japan, China, France, UK or Brazil (and many others) also have worthwhile offerings. You will find useful links to those data sources by searching the web.

Best regards, Michel.

A dataset in machine learning is, quite simply, a collection of data pieces that can be treated by a computer as a single unit for analytic and prediction purposes. This means that the data collected should be made uniform and understandable for a machine that doesn't see data the same way as humans do.

Most data can be categorized into 4 basic types from a Machine Learning perspective: numerical data, categorical data, time-series data, and text.

training set—a subset to train a model. test set—a subset to test the trained model.

The “training” data set is the general term for the samples used to create the model, while the “test” or “validation” data set is used to qualify performance. Perhaps traditionally the dataset used to evaluate the final model performance is called the “test set”.

Dear Ranjan Kumar Sahu,

If you have sufficient datapoints. I would suggest you to analyze the period before and after gap separately.

A followup question: Doesn't BERT rely on context or embeddings of nearby words to produce embedding of a word?

The following is a list of some datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some more datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some more datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

The following is a list of some more datasets which might be helpful for your work.

Dataset #1

MonkeyPox2022Tweets: The First Public Twitter Dataset on the 2022 MonkeyPox Outbreak - The dataset consists of a total of 255,363 Tweet IDs of the same number of tweets about monkeypox that were posted on Twitter from 7th May 2022 (date when the first case of this outbreak was detected) to 23rd July 2022. Link to the dataset - https://doi.org/10.5281/zenodo.6898178

Dataset #2

Twitter Conversations about the COVID-19 Omicron Variant - It presents a total of 522,886 Tweet IDs of the same number of tweets about the SARS-CoV-2 Omicron Variant posted on Twitter since the first detected case of this variant on November 24, 2021. Link to the dataset - https://doi.org/10.5281/zenodo.6893676

Dataset #3

Twitter Chatter about Online Learning during the Current COVID-19 Omicron Wave - The dataset comprises a total of 52,984 Tweet IDs of the same number of tweets about online learning posted since the first detected case of the Omicron variant. Link to the dataset - https://doi.org/10.5281/zenodo.6837118

color and clarity

check this link

you can also use dfa, numpy, scipy and sklearn.

I used those libraries in my EEG signal filteration and classification project

Mohamed Amine Ferrag, Othmane Friha, Djallel Hamouda, Leandros Maglaras, Helge Janicke, "Edge-IIoTset: A New Comprehensive Realistic Cyber Security Dataset of IoT and IIoT Applications for Centralized and Federated Learning", IEEE Access, April 2022, DOI: 10.1109/ACCESS.2022.3165809

check this out:

NVIVO

Also the text analysis engine in the IBM Modeler suite (similar to the one in PALANTIR) but much less expensive.

Richard E. Gilder, RN MS

Bioinformatics Scientist

The Gilder Company

TTUHSC Adjunct Faculty

@shahab : thanks. I normally use SAS myself. It is a friend's research that she just incorporated me. She uses SPSS!

Sorry for replica, there were errors in the webpage..

Younas Masih Have a look:

1. https://github.com/rli012/BioinfoHub

2. https://github.com/Vedant-S/Gene-Cancer-Classification_Principal-Component-Analysis

3. https://github.com/topics/cancer-diagnosis

4. https://github.com/topics/cancer-genomics

5. https://github.com/haniffalab/FCA_bone_marrow

Israa K. Salman Take a look https://github.com/ColumbiaDVMM/ColumbiaImageSearch

Souvik Panda Each item (typically nodes) in our network is given a fixed length vector representation by a graph embedding. These embeddings are a lower-dimensional representation of the graph that preserves the topology of the graph.

Sajal Das Take a look:

1.

2. https://ieeexplore.ieee.org/document/9225565

3. https://github.com/shaoncsecu/BanglaEmotion

Giulia Trauzzi Take a look https://academic.oup.com/bioinformatics/article/28/20/2615/201922

Dear Collin Nisler ,

There is a lot of things to consider. You have ~2000 sequences, but: are these sequences orthologs? Are there paralogs? Do you know the species tree for the organisms from which these sequences were obtained? Is the taxonomic sampling of sequences balanced across the species tree? What is the mean sequence length? What is the MSA length? What is your ASR approach (maximum-likelihood, Bayesian, or parsimony)? Those gap positions are due to true insertions and deletions (indels), or are some of those gap positions caused by incomplete sequences? If they are indels, have you considered to perform a MSA using a statistical model for indels (Poison Indel Process) as implemented in ProPIP MSA software? Have you compared different MSA softwares? Are these gaps present in important and conserved sites or are they only in variable sites?

I believe that the approach to dealing with the trade-off will vary depending on the answer to each of the above questions.

For instance, if you will use maximum-likelihood or Bayesian approaches for ASR, you should be careful with gap positions because the stochastic substitution models used in these approaches do not account for indels. If there are many incomplete sequences, I would remove them as much as possible. If the taxonomic sampling is biased toward one specific clade, and if many sequences of this clade are gappy, I would prune this clade.

I think that perhaps the closest thing to a general rule of thumb is to worry about the quality of your taxonomic sampling rather than the quantity.

Best wishes,

Pedro

Hello Irfan Ripat,

as you already gussed it is not a normal situation to obtain a better accuracy on test data in comparison with the training data. However, there are some things that you can test that help you figure out this situation. The two most conventional ones are:

Besides the above-mentioned plausible reasons, I searched the similar issue and found the following links useful:

I hope it helps you!

Giacomo Benvenuti Take a look https://www.mdpi.com/2306-5354/8/2/26/pdf

The relative weight analysis addresses the multicollinearity problems and helps calculate each predictor's importance rank.

Under the assumptions of missing at random (MAR) or missing completely at random (MCAR) data, you can use multiple imputation. Alternatively, many other software programs offer full information maximum likelihood estimation which can be applied to many common statistical procedures such as regression and ANOVA and relies on the same assumptions as does multiple imputation.

Thank you very much for your help but I am looking for multispectral or hyperspectral skin images, however the ISIC images are of the RGB type. Thanks again

Maybe try it on Github.

You can start with Linear regression, and then introduce the nonlinearity in variables not coefficients - like using the Polynomial features from sklearn library - Python.

In other words, Given 4 variables as X1, X2, X3, X4 - you can start with first degree LR => Y ~ X1 + X2 + X3 + X4; then move on to second degree without and with interactions => Y ~ X1 + X2 + X3 + X4 + X1*X1 + X1*X2 + X1*X3 + X1*X4 + ... + X4*X4; then move on to introducing nonlinear kernels => Y ~ exp(X1) + log(X2) + (1/X3) + ... something like this.

Once all these fails to capture the variations in Y data (which can be determined to an extent from the residual plot) move on to the other non linear techniques like PCR/PLS, SVR etc.

I don't use python for this instant, so I'll be suggesting a really simpler software.

Try Orange. It's simple as it doesn't need coding and extremely flexible.

You will have to develop a new model based on new variables then only test it.

Perhaps you may check some of other approaches to compute MSE:

The mean of the dataset is at around 32, and my RMSE is 1.70, is it acceptable then? @Medhat Sir

Dear Eric Dolores ,

Compound databases of natural products have a major impact on drug discovery projects and other areas of research. The number of databases in the public domain with compounds with natural origins is increasing. Several countries, Brazil, France, Panama and, recently, Vietnam, have initiatives in place to construct and maintain compound databases that are representative of their diversity.

Regards,

Shafagat

Hello Samaneh,

In R, there are usually multiple ways to accomplish a given task. For your query, consider using the match(a, b) command.

Below is sample R code implementing this with two demonstration data.frames. All variables are treated as strings here, so if your variables include other types, be sure to modify the code accordingly. Any unmatched cases in the original data.frame will have "NA" values for the appended variable.

Good luck with your work.

# pick matching cases across two data.frames, add a variable from

# second data.frame to first data.frame

# for simplicity, all vectors are treated as string variables

# so be sure to modify code if numeric, boolean, or other variable types are used

# sample data.frame to be augmented

df <- data.frame(make=c('Chrysler', 'Fiat', 'Ford', 'Ford', 'Volkswagen'),

year=c('2001', '2010', '2016', '2019', '2020'),

mileage=c('23.1', '33.4', '26.4', '15.5', '28.0'))

# sample of "larger" data.frame from which new variable (displacement) will come

df2 <- data.frame(make=c('Audi', 'Chrysler', 'Dodge', 'Fiat', 'Ford', 'Volkswagen', 'Volvo'),

year=c('2015', '2001', '2007', '2018', '2019', '2020', '2021'),

mileage=c('24.7', '23.1', '18.0', '33.4', '15.5', '28.0', '26.5'),

displacement=c('2.8', '3.6', '5.4', '1.6', '5.0', '2.5', '2.5'))

# show original data.frame

df

# show data.frame to be searched for the displacement variable

df2

# add new vector to original dataframe, for cases which match

# note that all unmatched instances in original data.frame will have "NA" values

df$litres = df2$displacement[match(paste(df$make, df$year, df$mileage), paste(df2$make, df2$year, df2$mileage))]

# show revised original data.frame

df

With smaller samples (depending on what you have as a population) you can try to validate your model with additional data collection especially when you are modelling social phenomenon that are highly fragile thus exhibit unexpected variances. I would not suggest a fixed sample size here but advise that you continue expanding your validation sample until the goodness of fit changes in response to new data is insignificant. Of course this assumes a very good sampling design and strategy.

Thanks

Please look at: https://www.google.hr/search?q=Multiple+correspondence+analysis&authuser=0&ei=LrG9Yp6oF4CI9u8Pq8OwsA4&oq=correspondence+analysis+and+factor+analysis&gs_lcp=Cgdnd3Mtd2l6EAEYBDIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwAzIHCAAQRxCwA0oECEEYAEoECEYYAFAAWABgs4UBaAFwAHgAgAEAiAEAkgEAmAEAyAEIwAEB&sclient=gws-wiz

SPSS has correspondence analysis in its manu too.

If you don't have any reference image in your dataset, then it is problematic, you are right, you can't use that function. What about to apply FFT to the images? You can have a look on the distribution of low and high frequencies. Low amount of high frequencies can indicate that the image is blurry, but you would need to estimate the right threshold of the "low amount" of the high frequencies. Maybe this approach is worth a try :)

Dear Shivreet Kaur ,

In recent years, RNA sequencing (in short RNA-Seq) has become a very widely used technology to analyze the continuously changing cellular transcriptome, i.e. the set of all RNA molecules in one cell or a population of cells. One of the most common aims of RNA-Seq is the profiling of gene expression by identifying genes or molecular pathways that are differentially expressed (DE) between two or more biological conditions. This tutorial demonstrates a computational workflow for the detection of DE genes and pathways from RNA-Seq data by providing a complete analysis of an RNA-Seq experiment profiling Drosophila cells after the depletion of a regulatory gene.

Regards,

Shafagat

Hi, I don't have a data set, but an interesting question. Try looking at relevant papers and contacting those researchers, they may share data with you if you collect your own.

An example might be:

Renzoni, A., Pirrea, A., Novello, F., Lepri, A., Cammarata, P., Tarantino, C., ... & Perra, A. (2018). The tattooed population in Italy: a national survey on demography, characteristics and perception of health risks. Annali dell'Istituto superiore di sanita, 54(2), 126-136.

Good luck with your research!

You use the terms "compare" and "correlate". Those are two very different activities (one is useful for answering questions like "which is bigger", while the other is useful for answering questions like "does variable A increase as variable B does?"). Without using either term (or any statistical term at all, hopefully), what is it that you're trying to find out

_____

The number of observations does not really matter. To regroup your lines as meaningful groups (or 'patterns', which might be seen as distinct variables in a way), you need a metric, a way to measure how two lines differ from one another. For instance, you may consider a score taking into account all the columns of a line; in general, one often uses the Euclidean distance, but it will not be the best in your case. Anyway, the idea remains to find a way to automatically compare two lines: this is where your knowledge in Business and Economics intervene.

Check this out: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiG0-uH4M34AhVoxYUKHcIoD6EQFnoECBkQAQ&url=https%3A%2F%2Fwww.econstor.eu%2Fbitstream%2F10419%2F73774%2F1%2FIfoWorkingPaper-118.pdf&usg=AOvVaw2s7COMAs8GxtjdDw_TJzIt

Citing under References is enough but if you want to publish any data table you need to take permission from the publisher

Hello there! What kind of predictive maintenance algorithm are you using in your application?

I can see why that article is no help, because it does not provide an citation for its method. As far as I can tell, "correspondence factor analysis" is simply correspondence analysis.

Why don't you try reposting your question and include correspondence analysis as a search term?

You can try to get similar datasets from UCI machine learning repository or from Kaggle.

This page includes relevant info.