Data Management - Science topic

With global research spending increasing rapidly, and a growing emphasis on international collaborations, professional management of research is essential. University leaders around the world are starting to see the value of building a sustainable research office, along with the research manager role. However, in many respects these roles are still highly adapted to their local research and funding environments.

To understand the growing trends in research management, and how the research manager’s role is changing in light of global challenges, we interviewed senior representatives of Research Management and Administration (RMA) organisations in Africa, Asia, Australasia, Europe, North and South America...

Master Data Management (MDM) is the process of managing and maintaining a single, consistent, and accurate source of truth for critical data elements within an organization, such as customer data, product data, and other key data domains. Implementing an MDM solution can be a complex undertaking that requires careful planning, execution, and monitoring. Key Performance Indicators (KPIs) can be used to assess the success and effectiveness of an MDM implementation. Here are some examples of KPIs that can be used for evaluating an MDM implementation:

It's important to note that the KPIs for an MDM implementation may vary depending on the organization's specific goals, requirements, and industry context. It's essential to define and track relevant KPIs that align with the organization's strategic objectives and desired outcomes from the MDM implementation. Regular monitoring and review of these KPIs can provide insights into the effectiveness and success of the MDM implementation and identify areas for improvement.

I'm happy to announce that we are currently rolling out the new and much improved version of the software to our beta testers.

If you would like to give it a try please sign up at our new website www.thot.so, or send me an email at brian@thot.so

(Jan Vališ sorry for the extraordinarily late reply, I haven't checked this thread for quite a while.)

Please contact Tactis.fr

Dear António Brandão,

As far as I know, post doctoral is self-guided research activity. One must do it to extend the PhD work.

Just do logistic regression is what I had in mind. The DV might be antcancer activity (yes /no) same for antioxidant activity. Best wishes David Booth

Ofcourse, you can use thread or Q/A feature on Researchgate to announce and call for beta testers. I believe, Researchgate isn't the best for such and other social -academic/software forum and also GitHub forum groups and sites are available. Even on this thread and on LinkedIn, and stackexchange

All SAS-related poster/presentation may be found at https://www.lexjansen.com/ Use their search engine for anything related to SAS.

I would caution against MANOVA because it's not what you think it is.

A lot of people assume that a MANOVA works like an ANOVA but with several outcome variables at once. This is not in the least bit true. MANOVA mashes your outcome variables together into composite outcome variables. And furthermore, it does it in a way that you cannot control. Instead of giving each outcome variable equal weighting, or even allowing you to decide on the relative importance of each variable, it basis the weighting on the relationship of each variable to the predictor variables. As Kendal Smith puts it ANOVA and MANOVA do not analyze the same variables and, thereby, address different research questions.

See

Huang FL. MANOVA: A procedure whose time has passed?. Gifted Child Quarterly. 2020 Jan;64(1):56-60.

Smith KN, Lamb KN, Henson RK. Making meaning out of MANOVA: the need for multivariate post hoc testing in gifted education research. Gifted Child Quarterly. 2020 Jan;64(1):41-55

Decision making in companies and organizational leadership should be dependent on the key variables that determines the effect of the decision to be made. The volume of data (big data) may be of help but is not decisive for effective decision making. The top management should identify the point that require decision (purpose), what key factors are related to the purpose and what resources are required to make decision for that particular issue. Relevant data on each of these variables is most decisive than the bulky of data. Nonetheless, the big data is of importance in reviewing the over all performances of the company and effectiveness of the leadership. For specific decisions, specific but relevant data is required. So the level (scope) of big data that affect decision making process depends on the level (scope) of the decision that is to be made.

epi info is free and has backup help

Hello Frank Krüger, thank you for your feedback, we recently started testing the eLabFTW in the workflow at our Electron Microscope. It would be cool if we could compare/exchange our methodology. It would be useful to consider a user meeting on this topic.

Oops sorry but I confused different Dr..Sharma's. The one I know has published in Decision Theory and Management Science quite heavily, but this one has a great deal of management experience. Mea culpa and apologies to all for my error. David Booth

I agree with Cenk. Directly sending emails to authorities may be a good choice based on my own experiences.

The use of Big Data Analytics database and analytical technologies can significantly improve the information offer of digitized online libraries. Analytics on digitized library resources using Big Data Analytics can significantly increase the possibilities of research in the field of book science. The results of these studies posted on the library website can significantly improve the information offer of the library website.

Regards,

Dariusz Prokopowicz

Thanks for your contribution and support @ Dr. Mohammad Ghaderzadeh

I like that, is great

One risk is connected with security issues.

But organizing a data storage using the I4.0 principles gives more flexibility and the security part is a price to pay.

We've been developing an approach to design a data management system for measuring forecasting accuracy across many time series and across multiple horizons and forecasting origins. By following the I.4 principles the following framework has been developed:

The framework presented implements the so-called Forecast-value-added (FVA) analysis (see slide 6) while following the principles of interoperability, decentralization, real-time processing, service-orientation, and modularity.

Our general problem definition is given on slides 4-5. The I4.0 principles helped obtain a cross-platform solution for forecasting, which is simple to implement and learn, and fast in operation (applicable in production settings).

Why you don't try Matlab

Jimoh Mohammed Idris - See examples below.

Most of the links that Mary provided are to rather general discussions of NVivo, rather than to technical problems like you are experiencing. My strong impression is that NVivo is quite solid, so I would try to connect with the support team at QSR to get a resolution to your problem.

It will be hard to conduct group therapy virtually!

No. You could try to standardize the simulation software but there are advantages and disadvantages to each. Your researchers are currently trying to pick the best software for their problems. If you want to maximize good research you let the researchers decide. Other than that, what is your goal? As a retired professor I can assure you that only allowing one statistical package on campus has never worked because not every package does everything. The same is true for simulation. The reason I use R now is because my university mandated SPSS which was of no help in my research. Thus I moved to R and I'm glad my university did that because I never would have discovered the power of R without the SPSS mandate. Other departments required everyone to purchase their department's favorite software. R is not only the most powerful it is free and open source. The question you are asking is really, which set of problems do you want to live with? With R I don't have either since I can ignore the mandate because of Rs power and cost. I recommend don't standardize(mandate) a single package for everyone. If you do then your simulation situation will be like our statistics one was. May the force be with you. David Booth

Possibly non sql DBs like MongoDB provide object based programming/scripting facilities for data storage and retrieval. It is the task of the developer to write script for optimization with respect to storage and retrieval. Some inbuilt function are there to move forward with optimization tasks, I think.

on paper

Hi Pohammad,

Big data/data mining, business intelligence/analytics, data science, distant reading, knowledge discovery etc. are all different terms used in different disciplines to denote essentially the same thing: statistical analysis and discovery of novel patterns from data and presenting them in the form conducive to human consumption. Meliha

Thanks, clear!

Hi Anna, RedCap ( https://www.project-redcap.org/ ) should have all you need for simple clinical trials.

Thank you everyone. All the answers were really helpful to cross my hurdle.

Cloud era

QGIS

Dear Frank,

try to open the Landsat images in ArcMap via "Add Data" --> Browse to the unzipped folder --> chose the "..._MTL.txt" that has a raster symbol.

Arcmap will now recognize the different bands and will stack them. You can now export the data and save them as a composite.

If you downloaded the uncalibrated data you can use Arcmap in addition to perform the Top of Atmosphere (TOA) Correction. To do so, open go to "Windows" --> "Image Analysis" --> select the data, e.g. "Multispectral_LC8..." --> "Processing: Add Function" --> rigth click "Stretch Function" --> "Insert Function" --> "Apparent Reflectance Function" --> Check "Albedo", --> In "General" tab chose "32 Bit Float". Now save and export the new raster, named for example "Func_Multispectral_LC8..."

Some links that migth help:

" Technology-based cognitive training and rehabilitation interventions show promise, but the findings were inconsistent due to the variations in study design. " [Ge & Zhu & Wu 2018]

If the datasets are big, then it can be happened. What type of RS images you are using MODIS, Landsat or Sentinel?

Thank you Professor Ette Etuk . As Ethiopia is somehow new to the technology, i wanted to generalize the management framework of big data, not just the analytics. I understand the analytics is an important part of big data but i wanted to start from the beginning Is there such a method that i can recommend as a policy guideline??

Basically anything you want to predict, estimate can be used in Construction industry. Some particularly application like: Fault detection (before/during/after construction); recommendation on architecture, color, etc.; price/construction time/resource prediction, when is the best time to start construction etc.; 3D reconstruction from 2D architecture, and many more. Many of these applications are related to computer vision.

It seems like some recommendation system are more related to your background.

Disclaimer. I am not expert in this topic but maintain a general interest on AI's applications.

Usually, unless you're perfoming time consuming operations like multiple imputation of missing data, loading the original data and doing all data transformation "on-the-fly" (to repeat all manipulations) is done very quickly, especially with packages like dplyr, tidyr or data.table.

If you really want to save interim steps, simply save the next steps in another object (data.frame), and finally save your complete workspace. There's a short, helpful SO-thread on this topic: https://stackoverflow.com/questions/34539011/saving-in-r-studio

I really recommend 1) using projects; 2) importing the raw data; 3) doing all data wrangling (manipulations) by writing a clean, commented script. Optionally, 4) save your workspace. Unlike other statistical software packages, R has the advantage that you're less often in the situation to save all your changes, as beginning from scratch by running your script is very fast.

Please refer to:

1. Landau, S. and Everitt, B. S.(2004). A Handbook of Statistical analyses using SPSS.

2.Howitt, D. and Cramer, D.(2008). Introduction to SPSS.

Regards,

Zuhair

Hi Allen,

With your credentials and background, you have plenty of opportunities. Having been in both industry and academia myself, I think your background is commensurate with both paths. If you consider industry, then you might look at corporate research and development (R&D) or some similar occupation where research and statistics are necessary, such as market analysis. Outside academia, you might also try to align yourself with a research-based think tank and pursue grant funding.

Good luck!

--Adrian

actually it depends on data , what kind of normalization n management do you want to apply. The basics can be taken from :

research

Do you want to study that how hospitals in Tanzania are using data management system within hospital and how they share or you want to study that how to start this in hospitals of Tanzania?

Thanks, Himadri! It's good information from China.

nice information thank you

nice information thank you

You can check the link below:

http://www.dataforensics.net/products.php

Ahmed - this question has been answered. Not sure why you make your statement. The book can either be loaned or purchased.

Use Qiqqa, it has autotags as well as multiple tags for you to do your own coding. It will automatically suggest themes! It also does Brainstorms.

The concern is that the pilot samples are (in some way) sampling a different population than the full study. Maybe the pilot survey was done just prior to a very successful national ad campaign. The campaign changed the proportion of people that use that type of product. Say the survey was on those people who drink orange juice. The pilot survey was sampling a population where 80% of the people had at least one glass of orange juice per day. After the campaign, half the people that used to drink orange juice now drink cranberry juice. So the pilot and full surveys have very different populations.

One solution is to break the analysis into parts.

1) Analyze the full survey as one part.

2) Reanalyze the data with the pilot survey results included. Add a blocking variable coded as 0 if there is no pilot data for that area, 1 for pilot data, 2 for survey data where pilot data is also present. I would delete block=1, and ask if there was a significant block effect (is block=0 any different from block=2)? I would delete block=0 and ask if there was a significant block effect. If the answer in both cases is no, I would combine the data.

Hi Ifeanyichukwu, this article appears really interesting and I made a request to get the full text. Thank you for sharing. Have a nice day. Regards

Found it: "AnnotationBustR: An R package to extract subsequences from GenBank annotations" Here: https://peerj.com/preprints/2920.pdf  

Dear Leanne,

I was in your situation 20 years ago with the same number of companies (20) and mixed data (interviews, observations, financial reports and survey results).

There are several major elements to handle the information

1) first, you create a standard SPSS file presenting each interviewee as a separate case (line) and adding some specific variables obtained from interviews and common variables for interviewees from the same companies from observations and documents. In this way, you will have the possibility to run different statistical checks on concordance/disagreement of opinions of interviews on the same topic, and also to include all data about a particular interview (date, time, etc.)

2) to make such a database useful you must create specific variables to present your impressions from observations and general results from interviews  as numerical variables. I made such variables as "SOC" for my impressions from interviews with CEOs on their preoccupation with employees needs, GOV for my impression from interviews with CEOs on their relations with local authorities etc.

3) Equally important is to design specific metrics for particular actions that reflect your general impression on a NGO resulted from the whole set of your data. For this you may look at the enclosed file. 

Success!

Igor Gurkov

I presume you mean 3 x 3!

When we extend the McNemar test, there are two hypotheses we can test. The first is asymmetry - that the off-diagonal values are asymmetric. The second is marginal homogenity - a test that the frequencies along the table margins are similar.

In genetics, the test is called transmission/disequilibrium test (TDT) and is used to test the association between transmitted and nontransmitted parental marker alleles to an affected child. I don't know if SPSS does the test. For sure, Stata and R will do it.

when using 0 1 coding, like genmod, the main effect only show the difference among the reference group, when interaction is used. If 1, -1 coding, the main effect value is not the difference with reference group, you need find the parameter by sum of others levels than multiply by -1. You can use contract+estimate option to get overall comparison as Firas's example, or conditional comparison. The main effect for categorical measure difference, the two ways interaction measure difference of difference, if the difference by one factor the same across the level for other factor. Three way interaction measure difference of difference of difference.

You can use the egen command in stata.

Can you please give me the English syntax. This may help me provide some more assistance....

I mean something like:

[Sn,] A Duration

Hi Izzeddin A. Bdair,

Questions similar to this has been addressed and you may like to read the following two:

Hope it helps

Regards

I do have a question on no Parametric method. If we use MCF method how to estimate the future failures and what will be the distribution . Basically I want to do system modeling using monte carlo. so my excel sheet is based on reliability equations which is different for each distribution

Dear Rajkumar,

I agree with Rudolf. It depends on the distribution and the statistic you are trying to obtain.

As per my knowledge, for climatic variables, the period of 30 years is considered as a standard to assess the changes. For Mann-Kendall non-parametric test, which is used to calculate trend, if the no. of years is greater than 10, the test assumes it to be normal.

Furthermore, when you go for normality tests for small dataset, I have observed from various literatures, if your n > 50 you should use kolmogorov-smirnov, and if n<50, should go for Shapiro-Wilk. Again, as described by Rudolf, the rate convergence remarakably varies for different distribution also.

You can refer the book "The SAGE Encyclopedia of Social Science Research Methods" for various statistical queries.

I hope this helps.

All the best !

Dear Maria Jose Monsalves 

please check the resources

###  Check over the following for any errors before use

### Sample data from http://rcompanion.org/documents/RHandbookProgramEvaluation.pdf

if(!require(geoR)){install.packages("geoR")}

Turbidity = c(1.0, 1.2, 1.1, 1.1, 2.4, 2.2, 2.6, 4.1, 5.0, 10.0, 4.0, 4.1, 4.2, 4.1, 5.1, 4.5, 5.0, 15.2, 10.0, 20.0, 1.1, 1.1, 1.2, 1.6, 2.2, 3.0, 4.0, 10.5)

hist(Turbidity, col="gray")

library(geoR)

T.box = boxcoxfit(Turbidity, lambda2=TRUE)

lambda = T.box$lambda[1]

lambda2 = T.box$lambda[2]

if(lambda==0){T.norm = log(Turbidity + lambda2)}

if(lambda!=0){T.norm = ((Turbidity + lambda2) ^ lambda - 1) / lambda}

hist(T.norm, col="gray")

There are several ways to interpret this question. I will assume that you are generally familiar with the effect of sample size on power, accuracy, specificity, type I and type II errors, and so forth. You would have had to go through this when planning the original sample size.

There are many good things that happen with increased sample size. There is one bad aspect to increased sample size. With the larger sample size you are able to detect smaller treatment effects. So you can find a statistically significant difference that is too small to be biologically meaningful. However, there is seldom enough information about the system being studied to be able to balance the treatment effect against all other sources of biological variability. Typically biological experiments in the lab are too controlled, and this can result in unexpected field results following promising laboratory experiments.

Maybe you want to quantify the benefit realized from increasing the sample size. In this activity I would resample the data and run the analysis with several thousand random selections of 79. You now have a distribution of what might have been the outcome if you had stopped at 79 replicates. On this graph plot the results using all the data to show the effect of having increased the sample size. A less powerful approach would be to compare the results from the first 80 to the results using all the data. Had things worked differently, one would presume that you would have simply stopped with the first 80 in the original design.

In terms of formulating the null hypotheses, increasing sample size will have no effect. However, sometimes increasing the sample size can enable you to look at factors that you would have had to ignore with the smaller sample size. Are there differences between females and males, ethnic differences, age, weight, and so forth.

Did you means something else?

its done now. I need to know if anyone has used ASI data of eyar 1994-95

Dear Mohammad

Go for CloudSim. You can simulate most of the aspects required with respect to cloud in  that. But there are some limitations in CloudSim with respect to the working when we compare with real cloud environments.

Best Regards

Quick and dirty approach:

1. estimate the value of a: it is equal to H at very high T

2. plot log(-log(1-H/a)) vs. log(x)

You should see a straight line.  If not - adjust (increase) a slightly until success.  The slope is then equal to parameter c, as the straight line equation reads: log(-log(1-H/a)) = log(b) + c*log(T).  The value of b may be found easily then, too.  But don't ask me what are the uncertainties of so obtained a, b, and c.  Anyway, it should be a pretty good start point (initial guess) to more advanced computations.

Thanks, Stephen. I will let you know what I discover.

Hi,

You can use INDEX to test for substrings.

COMPUTE PI = INDEX(POSFEED, "PI") > 0.

At some point of time "yes", please refer to "CAP theorem" (https://en.wikipedia.org/wiki/CAP_theorem) and various variations of consistency, especially you will be interested in "eventual consistency" (https://en.wikipedia.org/wiki/Eventual_consistency).

Ok thanks a lot Dr Bento

How the experts think the items will perform matters less than how they actually did perform. I would recommend removing them.

Dear Sanjay Garg,

İ found inreresting articles and sent to you.

Regards, Shafagat

Hi Karen

If your main interest is prevalence estimates that are generalisable to the target population, then you should really undertake a weighted analysis - even if it costs more. If your main interest is associations between variables (typically using some form of regression analysis), then weighting will make little difference to the results.

Adrian

I have not. Thanks for the suggestion. I will look into it.

Here is an interesting open-source data portal platform:

Thank you for the clarification.  So you are asking how to transform one variable that currently has a non-normal distribution using a logarithmic transformation in the hopes that it will give an approximately normal distribution.  The necessary SAS code is as follows:

DATA dataset2;

SET dataset1;

LOGVAR=log10(VAR);

RUN;

This would create a new variable LOGVAR that is the log10 transformation of the variable VAR. 

How you calculated transmit time, then I can exactly answer. Receive time would also have same things to be considered(hops processing, queuing, ready to transmit, recevie device acceptance time etc). I need to know what exactly you want? If you tell me how you have calculated transmit time then I can answer the exact thing.

As far as I remember, regression trees are very robust to outliers and skewed distributions in EXPLANATORY variables. For the response/dependent variable, that's a different story... Transformations can make sense.