Data Analysis - Science topic

The data can still still be used for analysis and any conclusion drawn should be in references to the period / time the data was collected.

In short, it all depends on your research questions, research problems & research objectives

Thanks!!! I will try once more.... with your suggestion.

so nice of you Sir

Thanks

Detecting and mitigating fraud requires two things:

1. Identifying normal patterns

2. Identifying deviations from the norm

The specifics will look different depending on what type of data you are looking to detect and mitigate fraud in. Let's take financial banking as an example. In this field, the size of the transaction is one of the largest indicators of fraud. Researchers have determined that the majority of transactions are relatively small. They have also determined that the majority of fraudulent transactions are large (given that it is more profitable for the fraudulent actor). Thus, one factor financial fraud mitigation algorithms look at is the size of the transaction, seeking to identify transactions that are outside of the normal range. Of course, there are hundreds of other factors that go into these algorithms as well. However, the premise of almost all fraud detection and mitigation can be broken down into those two steps.

In terms of the specific technologies used, Bayesian filters are very common. This is the form of algorithm used in email spam detection. Machine learning models are also very common, particular for detecting and mitigating fraud in big data.

Hi Adnan,

Here are a few common ones:

It's important to note that the choice of normalization method should depend on the nature of the data and the decision problem at hand.

you may include the other realted of parameters that have a main purpose for the urban agriculture in different measures .

It depends you research questions and nature of the variables.Identifying these you can apply the appropriate non parameteric test. For each parameteric test there is a counter part non-parameteric. so, choice the appropriate one for you data type nature.

Secondary data analysis is a research method that involves the use of existing data that was collected for a different purpose or by another researcher. While it may involve original research and analysis, secondary data analysis is generally not considered a unique study in the sense that the data being analyzed has already been collected and is not unique to the researcher conducting the analysis.

Instead, secondary data analysis is often used as a way to answer new research questions or test new hypotheses using existing data sources. It can be a valuable method for researchers who want to study a topic that would be difficult or impractical to study using primary data collection methods, or for researchers who want to compare their findings to those of previous studies.

That being said, secondary data analysis still requires careful planning, analysis, and interpretation, and can lead to novel and important findings. Therefore, while it may not be considered a unique study in the traditional sense, it is still a valuable research method that can contribute to the advancement of knowledge in a particular field.

There are other choices in the same verin as Bruce suggested. I attached one I found useful. Best wishes David Booth

Yes like Omar said, it should work. Also, check your format for lon and lat are correct according to the manual.

IMO R is the best. To get started with R see the book R for Everyone by Lander which contains research grade code. Best wishes David Booth PS there are many more books but I found this one to be best for me. It included download information too.

Hi. It would depend on the type of data you have and the sample size. There are many approaches depending on what you wish to find out. For example, you have frequency statistics that is for basic descriptive presentation. Then you have inferential statistics such as Correlations to determine if there are directly proportional relationships between variables. Factor analysis can reduce data to common factors. The more complex one is Structured Equation Modeling, and this is used to 'Test' a theory through your results.... these are just some. There are many more.

Results are liable to be highly biased. Those who respond and respond quickly may represent a stratum or subpopulation which may be quite different from the unknown population in general.

If you have good covariate data, you might use that. See "Comparing Alternatives for Estimation from Nonprobability Samples," by Richard Valliant, December 2019, Journal of Survey Statistics and Methodology 8(2),

DOI: 10.1093/jssam/smz003

https://doi.org/10.1093/jssam/smz003

https://www.researchgate.net/publication/337950671_Comparing_Alternatives_for_Estimation_from_Nonprobability_Samples

However, that seems unlikely.

Dear Kian,

have you tried improve your purity by FACS? It´s fairly easy to choose markers to distinguish MSC & macrophages and sort highly pure populations.

Laerd Statistics is considered gold standard to assist test selection, test procedure and interpreting and reporting results. This resource uses SPSS and STATA etc

Hello,

I just calculated the WQI number through pH, temperature, and TDS/TSS parameters in Microsoft Excel some days ago.

Would you like to try my formula in Excel file? I can send you through your email.

The Excel file is courtesy to my study in Water Quality Assessment from my University.

Thank you,

regards.

I would highly recommend you that before directly jumping into using certain tools for the analysis, please try to understand the basics behind the data, data types & structures, whys and whats of data processing. And for galaxy, it is a very good platform with very good tutorials. Please go through tutorial before asking question which can easily be solved with minimal self inputs.

It's very unlikely that you get such a strong defined band as a contaminant and nothing else (like the smear above) inbetween the three expected VP bands and this band. It could be an artifact of the gel system (dimerized or oligomerized VPs). Did you try with fresh DTT or b-ME in your SDS-PAGE loading buffer? Western Blot could show if it's VP or not.

I have found a workaround - export the native ".d" file and use ProteoWizard's MSConvert to convert it to a ".mzxml" file that contains all spectra.

It depends on your aim: if you want to test a model and show a model fit, AMOS is the best option. If your aim is exploratory study and theory development, PLS SEM is the best option.

Please be careful, small sample size is a myth in PLS SEM and researchers always should justify the minimum required sample size.

Please see the book of Hair et al. 2017.

Sharfras Navas Based on the title of your study, either Amos or SmartPLS could be appropriate for analyzing the relationships between knowledge, competency, adoptability, sustainability, and AI technology among physician entrepreneurs in GCC countries. However, the choice between the two software programs depends on various factors such as the complexity of the research questions, the sample size, and the nature of the data. Amos is better suited for analyzing more complex models and larger datasets, whereas SmartPLS is suitable for smaller sample sizes and more exploratory research. It is recommended to consult with a SEM expert or a statistician to help determine which software would be most appropriate for your specific research study.

Reynaldo Senra you are not explicit here, but there are several models, which could be termed "ANCOVA", since they follow the same principle, but in all cases, you need to know which value of your DV is paired with your CV (and of course to which independent group this subject belongs). But to clarify the term "paired data":

1) In your case (when you would have the paring information) you have a paired repeated measure AND between measure (a split plot design). Here, you have the pairing in your focal DV, which has been measured twice (t1 and t2). In case of a random assigment to the between factor, an ANCOVA with t2 as DV, Group as between factor and t1 as CV is appropriate and should have more power as a Split-Plot ANOVA with t1-t2 as within factor and Group as between factor.

2) Imagine a pure factorial between design, where the possible CV is not the focal variable, e.g. 1. factor sex and 2. factor experimental condition. Possible covariables could be income or any other variable you are interested in "to control for" (the problem here is beyond the scope of this topic but look for directed acyclic graphs DAG, if interested). Here is no paired or repeated measures design itself, but you still need to know the pairing of the DV and the CV of course, they have to be paired to account for the (co-)variance of the CV.

3) It gets more complicated if you have the split plot design mentioned above with a within variable and a between variable, but as in case 2) you want to account for non focal CVs. You could analyse your t1-t2 plus between factor designed with a split plot ANOVA AND incorporate an additional non focal CV. I would not recommend this, since the interpretation may get complicated, but it is possible in princinple. It would be something like a split plot ANCOVA, with a reapeated measurs factor, a between factor and additional, non focal CVs.

Therefore, if you want to incorporate CVs, they always have to be "paired" with the DV, but this is not the same as a "paired" or "repeated measures" design. I hope this clarifys it a bit and is in accordance to what you read so far.

Sharfras Navas

The choice of SEM model in Amos or SmartPLS depends on the research design, research questions, and hypotheses, as well as the type of variables involved in the study (e.g., continuous, categorical, ordinal).

Amos is commonly used for structural equation modeling (SEM) and can handle both confirmatory and exploratory models. It can also handle models with latent variables, and has a graphical user interface to specify models and inspect results.

SmartPLS is a software specifically designed for partial least squares (PLS) SEM. It is often used for predicting relationships between latent variables and measured variables in both small and large datasets. It can also handle models with formative and reflective constructs.

Both Amos and SmartPLS have their strengths and limitations, and the choice between the two will depend on the specifics of the research study. It is recommended to consult SEM literature and experts to determine the best fit for a particular study.

https://psyarxiv.com/ is common for psych, but if your goal is to get a lot of readers, with any of these it will be helped by posting links on social media. As since we are on ResearchGate, you can post here too.

All of the programs you mention are very effective for marking and retrieving segments of text. Since those are the core activities in most forms of content analyses, they are useful tools for those goals. I think there are reasonable questions about the application of these programs in more interpretive approaches, but less so for content analysis.

There are several causes for the presence of outlyers. Impurities, errors during the extraction, the needle that for some reason did not inject the same amount of sample, and samples that are naturally different from the other ones.

You can solve the problem using this checklist.

- check the look of the chromatograms and verify that all of them look more or less the same.

- check IS intensities and verify that their peak area is within an acceptable range for all the samples.

- normalize the data across samples.

- scale the data using UV scaling.

- evaluate the presence of outlyers using RobustPCA.

good luck

I agree as well. In my dissertation frankly the faculty was not entirely prepared to deal with a quantitatively analyzed thesis which actually required taking course work from another institution. There were not sufficient courses offered in statistics there to fulfill or meet the expected level of expertise offered to write such a thesis.

Open-Source fNIRS Tools

KARUNAMURTHY A

Open source fNIRS data analysis, Homer, NIRSLab or MNE?

All of the options you mentioned, Homer, NIRSLab, and MNE, are open-source tools for analyzing functional Near-Infrared Spectroscopy (fNIRS) data.

Homer (HyperOxford multivariate analysis of fNIRS data) is a software tool for processing, analyzing and visualizing fNIRS data.

NIRSLab is a MATLAB-based toolbox for processing and analyzing fNIRS data.

MNE (Meg/EEG and Neuromag) is a Python-based software for analyzing MEG, EEG, and fNIRS data, providing a comprehensive and unified framework for the analysis of neurophysiological data.

In terms of specific capabilities and suitability, it depends on your specific needs and preferences for data processing, analysis, and visualization. You may want to try out each of these tools and see which one works best for your particular use case.

I would start right away to teach data analysis using GNU R, without spreadsheets. There are good learning materials, as https://r4ds.had.co.nz/ (Tidyverse approach) or https://github.com/matloff/fasteR (base R). Generative AI is not relevant here in my opinion, first you need to understand data summarization, filtering, grouping, ploting, all the basics. Then statisitical inference. Maybe in your next courses it would be good for students to teach them SPSS / SAS / whatever is used in the industry. Spreadsheets are great for accounting tasks.

As Christian Geiser mentioned, you can adopt two approaches: investigating variables separately or collectively.

For the former, you can use non-paametric tests as well (like Mann-Whitney test). However, Regression is the most common model for the second approach.

Hi Saliha,

Maybe the MIQE guidelines can help you with some information about PCR/qPCR procedures.

I sending the link: https://pubmed.ncbi.nlm.nih.gov/19246619/

Have a look at this one which I've previously purchased:

Outlier is a fuzzy term and it helps to be clear what you mean. Personally I'd define outliers as observations that aren't from the distribution/population of interest. So checking for impossible values and data entries make sense. Some people define in terms of extremeness (which I think is not useful). Others in relation to a model - in which case outliers have extreme residuals rather than being extreme (its possible for an outlier in this sense to be too average/central rather than too extreme).

Generally there isn't much reason to remove outliers or potential outliers and I'd avoid it. If the problem is extremeness it may make sense to switch to a more robust model rather than delete data. More importantly I'd rather look at influence than the extremeness of a residual. Quantities like Cook's distance (other influence statistics are available) give an idea of whether an observation changes the parameter estimates in the model. Generally you want all Cook's distances to be below 0.5 and might worry if you have values above 1. In these cases it can be helpful to check the parameter estimates to see if the model and findings change much if you remove high influence observations. (I'd probably not delete these but report what happens in the alternative model as a potential limitation).

Quick summary:

1) Do check for anomalies/data errors

2) Consider whether there's a better model that fits the data with fewer extreme residuals/more reasonable assumptions

3) Look at influence rather than extreme values

4) Avoid dropping data but consider reporting alternative analyses with/without extreme/influential observations

To analyze data and create various types of plots in Python, you can use a variety of libraries and tools. Here are a few options to consider:

Update chromelon software or get originlab are your two best choices it seems to me David Booth

There are several research question and response forums that may be of interest to you, particularly if you are interested in phenomenology and Van Manen's method of analysis. Here are a few options that you might consider:

It is worth noting that while these forums can be a useful source of information and guidance, it is important to carefully evaluate the quality and credibility of the information that you find, as not all discussions and responses may be based on rigorous research or reliable sources.

Also, take a look at this page: https://www.smartpls.com/documentation/choosing-pls-sem/pls-sem-compared-with-cbsem

As I assume you know, with such a large value you have to reject the assumption of randomness (if you cared to test it).

Regarding any errors, only you can verify them by carefully reviewing the data matrix.

One typically would use an ANCOVA when one has a continuous independent variable, along with the categorical independent variable. For example, say you are studying the effect of a Drug upon recovery of patients. With an ANOVA you can study the recovery with or without the Drug. If, in addition, you have information about say about the Age of your patients, you can run an ANCOVA to study the recovery patients "controling for" Age.

An advanced data analysis method for time-to-event data (e.g., response times, saccade latencies, fixation durations, etc.) that replaces ANOVA on mean RTs, is a longitudinal or distributional method known as event history analysis, survival analysis, hazard analysis, transition analysis, and duration analysis:

Yes, qualitative software programs such as NVivo are frequently used in constructing literature reviews. In that case, each item in the literature review its treated as a document and and you can apply codes, just as you would with any text document. But NVivo would be expensive for this purpose, so you might take a look at either Mendeley or Zotero.

NVivo can create a schematic diagram that links any set of coded texts, but if all you want is a "concept map," then there are any number of free programs that will do that.

It is possible to collect the qualitative data from the non-sampled respondents like Key Informants Interview and Focus Group Discussion in the open environment of the villages or concern settings where these respondents have very indepth knowledge about the historical events and to recall quickly while asking the questions.

Having collected those data from these groups of people that you can enter the data in Atlas Ti-9 software will help you to bring the expected results for your research.

All you need to have a good practice as well as good rapport with those community could help you to get the validity and reliability of the research.

You are using count data so I suggest that you read the attached R information. I'm also attaching a paper of ours that uses similar methods for binomial data that may be of some help to you. If you have questions please ask. Best wishes David Booth

Usted esta comparando dos grupos de datos en cuanto a normalidad , usando el esatdistico de K-s , kolmogorov smirnov, , primero deben ser muestras grandes, mayores que 30, vamos a las hipotesis:

ho: las observaciones son normales

h1: las observaciones no son normales

sin hablar de alfa ni de los otros pasos que faltan en el test, si ambos grupos de datos son normales me quedo con el mayor p-valor , siendo ambos mayores que alfa, es decir, ambos p-valores deben ser mayores que alfa, para no rechazar ho, y luego me quedo con el set de datos que posea mayor p-valor, por que !!!, bueno esto lleva otro analisis mas profundo !!!!, pero sencillo, se lo puedo explicar , digame usted si desea la explicación !!!

Thank you all for responding to my question.

I would make a few additions:

When reading about masking technique, we can often see the descriptions like “ignore the missing value” or “skip the input” with masking. On the other hand, there are few references in the literature that explain in mathematical formulas the skipping of input missing values.

Then, I wonder what exactly happens inside the RNN layer when input is masked.

As you can see in the attached image of the formula, RNN state z^t in timestep t can be expressed with the input x^t, recurrent input z^{t-1}, and the wights W.

And, if the input is masked to be ignored, how is z^t calculated?

Is z^t calculated with imputing x^t (like, same values in previous timestep is inputted) ?

Or, is z^t not calculated and exported as NaN value?

Thank you very much.

Such questions don't make much sense.

Each tool or software has its pro and its cons, and it always depends on the special kind of problems you try to use a tool for. Generally, it does not harm to know a couple of tools to select the one that is best suited for a particular problem.

What is more relevant to a beginner to choose a (first) programming language is which language is typically used in the field and if there are accessible resources and collegues to get help from.

After knowing one programming language well, you start understanding the programming principles, you start thinking like a programmer, what considerably fascilitates learning another language.

Would you please tell us, how did you calculate nri and idi in stata?

Dear Ambil,

Do you mean you have some data and you need to transform the data using ilr-transformation? If so please read the following post, please

You need to install the package "comopisitons" in R software.

Any further question is welcomed.

Good luck,

Hamid

Yes of course, you have to use the proper weights. In my regression software this is done automatically. Just enter your measurements and their standard deviations (or any other estimation of the uncertainty).

According to Kolmogorov-Smirnov test and also the shap of plot. I think yes.

Conduct a literature review:

Following are a few acceptable sources for literature reviews:

1. Peer reviewed journal articles.

2. Edited academic books.

3. Articles in professional journals.

4. Statistical data from government websites.

5. Website material from professional associations

Have you tried using logariths or a gamma- or quasi-Poisson GLM?

If your the IV have sub groups e.g.

Age = in range (14-16, 17-19, 20-22)

Gender = m x f

GPA in range (1.5-2.00, 2.1-2.5, 2.6-3.00, 3.1-3.5, 3.6-4.00, 4.1-4.5, 4.6 and above)

Then you use 3 way ANOVA

But if there are no groups, you can use linear multiply regression

Best Camille Patrice de Leon Orpilla

I would say you must build a new quantitative method in "MS Quantitative Analysis" software. It takes some time to build a method, however, if you do it once then you'll find it easy to do it another time.

Analysis tool NVIVO is recommended...thematic analysis or IPA is also used for qualitative data analysis

longitudinal studies are conducted over reasonably long period of time. For example, in medicines and testing forecasting models. These are conducted when on spot or single time measurement of data not relevant like routine commercial surveys or experiments on some variables in laboratories where lapse of time is not essential to derive conclusions.

Long term side effects of medicines are one such thing. Another is creating weather forecasting models which need reasonable long-time data. Si what you have to do is change various forms/levels of control variables in your study over, say, a semester. In such studies control variables are essential to identify. Please first research literature to see what relations ships of various variables is exist so you can set up constructs. More can advise after seeing your further inputs

Ok, let's say that some statistics obtained from regression models are sometimes used to describe a sample... (the sample mean being the most frequently used). If you also include quantile regression, all sample quantiles and derived quantities may also be called statistics from regression models.

If we use a quantitative study, the sample size can be determined by using a certain calculation procedure taking into account the sampling error, the significance level and also the confidence level. We usually choose to use a sampling error of 5%, (significance level (α)- .05) with a confidence level of 95%. Usually to determine the sample size, we can refer to the sample size determination table that has been prepared such as the table of Krejcie & Morgan (1970) and the table of Cohen et al. (2001).

R is another option.

In my opinion, there's nothing that prohibits you from reporting values that fall below some "magical" threshold. Most of such cut-off values are more or less arbitrary anyway. "Negative" findings can be just as important as "positive" findings when it comes to scale evaluation.

What I see as more important is to figure out and explain in the paper why you are getting suboptimal values for your measurements. Perhaps the results indicate that the measures/scales are rather heterogeneous and/or multidimensional or that some of the items should be revised, dropped, or replaced.

Fellow researcher,

If you allow me the undeserved familiarity in my speech. Duuude, phyton and R are the best data analysis tools there is and both are free and have (freely available) tons of awesome tutorials.

There are 2 primary ways of handling missing values: Deleting the Missing values. Imputing the Missing Values.

Dear Iqra - Imtiaz,

Taking into account your research interests and scientific specialization, the discipline of your master's studies, I propose the following research topic for your thesis: Improving analytical methods based on Big Data Analytics used to analyze trends and forecast future behavior of citizens using the services of companies operating in the telecommunications industry and key information services available, among others on smartphones with Internet access. As part of this topic, you can use Internet user sentiment analysis in any specific field of Internet users' interests in combination with the research method based on Big Data Analytics and possibly also with the use of some other and available Industry 4.0 technologies. I described how to do it in my articles with the concept of Big Data Analytics in the title. These articles are available on my profile of this Research Gate portal. In addition, under the proposed topic, you can also explore the use of individual types of Industry 4.0 technologies, technologies typical of the current fourth technological revolution, including Data Science, artificial intelligence, machine learning, personal and industrial Internet of Things, cloud computing, web applications operating as robots, horizontal and vertical data system integration, multi-criteria simulation models, digital twins, additive manufacturing, Blockchain, smart technologies, cybersecurity instruments, Virtual and Augmented Reality, 5G, etc. in the field of improving Internet information services available from the level of a smartphone equipped with Internet access. On the basis of this type of research, you can point to those Industry 4.0 technologies that are likely to continue to be used in the next few years to improve Internet information services offerings. As part of this topic, you can also study the issue of development prospects for the continuation of the process of improving analytical methods based on Big Data Analytics with the help of the above-mentioned Industry 4.0 technologies.

Kind regards,

Dariusz Prokopowicz

What fraction of values below BDL is acceptable.

Why are you making the measurements? The why determines what is acceptable.

If you are concerned about an upper limit, then BDLs are of no concern.

If you are concerned about a lower limit, it will depend upon the nature of your concern.

There is no recommendation. There is no rule of thumb.

You decide from the criteria associated with WHY if you have enough information.

Too many BDLs might mean you need a different technique, but it always returns to WHY.

If , say, a customer wants an answer at each location, use the actual result and note the uncertainty. The result is usually meaningless, because of the high uncertainty.

You don't give a research question so I would simply report exactly what you said in full detail. You may want also to give information on the detection limits of the methods you used. Just let Prof Kan know detection limits are determined by Nature's own chemistry and not by personal choice. Best wishes David Booth

Hi Mathew, thank you for your suggestions! Yes, I used the Diagenode biorupture sonicator. I will go through your suggestions! Since I am a molecular biologist it takes some time to me to understand the bioinformatics related stuffs!

This list server has periodic announcements about online courses in qualitative research:

Thank you, Amogh Shukla.

While I appreciate the large EdNet hierarchical dataset, it is too complex for my use case (example dataset for R package). I would need a simple tabular dataset (i.e., observations/units in rows and variables/features in columns).

Hierarchical *clustering* datasets are not suitable unfortunately because I'm interested in hierarchical *classification* and clustering is an unsupervised methodology without an outcome.

Actually, the stackoverflow question you cite is by me, but the answer given there does not help unfortunately because, with the data in that answer, the outcome is not hierarchical, only the covariates are.

For now, I use a simulated dataset from the collection given in https://github.com/jona2510/ADforHC . However, it would be better to have a real dataset as an example dataset.

yes, I would have thought logistic regression, but I'm testing my memory here as it is over a decade since I did something similar