13th Sep, 2020
Adam Mickiewicz University, Poznan
Question
Asked 30th Mar, 2020
How do you deal with lost data?
Given your specific discipline. Have you ever irretrievably lost data of an ongoing research project? How did you handle it? Thanks in advance.
(Also, this is my story. A couple of years ago, in a study that included collection, preservation, identification and weighing of soil invertebrates, after an unfortunate event in the laboratory, the notebook that contained the weight notes of one of 10 sets of collected organisms, which belonged to the control group, was lost, so were the preserved organisms. I'm tagging this with an entomology lablel, so in case you're familiar with this topic: Would you consider trying some method of reconstructing the weight data or is there just nothing to do? There is no way to recover the notebooks, nor the preserved organisms).
Thank you.
Most recent answer
Hi,
I also lost a lot of data one time and in the result I have one paper published less :)
1 Recommendation
Popular Answers (1)
Javier García Reynaud, I first try my best to store it in multiple locations and drives so as to prevent any data loss. if it happens to occur, though, I use some recovery apps like "recuva" for instance.
4 Recommendations
All Answers (23)
The question arises: Does the issue of data loss concern the loss of data from memory drives, hard drives, SDHC drives etc. installed in computers, tablets, smartphones etc.? If YES, then use disk recovery software. If the disks have not been physically damaged or otherwise damaged, e.g. by fire, strong electromagnetic waves and / or have not been physically damaged in whole or in part, it is likely to recover all or part of the data from the disks, even when using the Internet. free programs to recover data from disks.
Regards
Dariusz Prokopowicz
2 Recommendations
Javier García Reynaud, I first try my best to store it in multiple locations and drives so as to prevent any data loss. if it happens to occur, though, I use some recovery apps like "recuva" for instance.
4 Recommendations
I stored my important data in many drives in order not to be get lost,and if happens I used recuva,as Amjed said😌
2 Recommendations
I think you answered your own question, the data is lost irretrievably and therefore you cannot recreate it however you try. As time has passed, even if you were to go back and collect new samples they would no longer be compatible with the rest of the data set due to possible intervening changes over time. Your only hope is to explore the components of the data set that still exists to see if it provides some ideas for a future experiment and hypothesises that you can postulate for study.
I have lost personal data files before. In some cases I have kept a log of the sources and have gone back to retrieve data files. Other times, it is just gone, and I must perform an manual recovery.
IF it is mission critical data, it might be stored in the recycle bin and recoverable as long as I haven't shut the system down or emptied the Recycle bin. Both Microsoft Windows (for sure) and various flavors of Linux have a recycle bin (pretty sure).
If I haven't written anything new, I might be able to run a file recovery program.
And lastly, I can send the hdd to a company that does drive restoration and recovery, and hope for the best.
Working on a government system, I can request a restoration from their daily back-up set.
Most times though, in the mindset of triple-redundancy, I routinely make off-system back-ups. If my data files can fit on flash-drives (16, 32, 64, 128 GBs), I carry a set of 3 with me, and back-up during and at the end of my workday.
If the data files exceed my available flash drive capacity, I have available external ssd/hdd drives in capacities of: 1, 2, 4, 5, & 6 TBs. I can perform a back-up when I am not using the system, AND routinely schedule FULL BACKUP whenever I am I am concerned about data loss which is all the time, actually. A full backup may become an overnight operation depending on the amount of data for backup.
None of this may help you in your present situation, but may keep you out of trouble in the future.
Good Luck.
Michael
1 Recommendation
Hello Javier; I am imagining that your records were on paper (you said notebook). If you study the remaining control data, would what remains still provide an honest control? If the answer is yes, then you can go ahead and use the smaller dataset. If the answer is no, then you are obliged to start over. What a pain!!! I am acquainted with a person whose dissertation manuscript was lost when her computer crashed...no backup. That's the worst! The lesson is that some kind of backup is needed.
In the early days of my career, in the early 1960s people used lab notebooks that had carbon paper between pages. When your work day was finished that copied page was taken home and stored separately from the notebooks in the lab. That was the ancestor of the external hard drive. It worked.
Best regards, Jim Des Lauriers
1 Recommendation
I have not lost the data of any ongoing research project, because in parallel I work at home and do independent archiving. Experiments have been suspended, which is a waste of time.
2 Recommendations
It is better to always keep multiple copies of the collected data and I have heard that hard drives can be safe place to keep your data as computer professional can retrieve data from the hard drive.
1 Recommendation
HDDs do fail. So, multiple copies are best.
I subscribe to the government mentality of triple redundancy.
Besides the copies on my PC, I maintain a set of backup drives, external drives.
I make a full back-up of my systems one or more times a week depending on the importance or criticality of the data if I lose it.
Additionally, if the data files are not too large, I will make multiple copies on an external flash drive that I can carry in my pocket.
1 Recommendation
Hello all; I think that Mr. Reynaud referred to "notebooks". I took that to mean paper ones. Most of the replies above are couched in terms of digital records. We don't seem to agree on what the questioner actually lost. It was a disaster in any event. Regards, Jim Des Lauriers
1 Recommendation
Javier, sorry to hear about your loss. Although it is advisable to have a back-up for collected data by saving in multiple areas and technological devices, I think you can try contacting your research participants again to see whether they will sympathetic in providing the same data again. You may also wish to tap into the expertise of ICT experts to see what help they can offer in retrieving your loss within specific devices where these may have been saved. You may also wish to secure the services of an additional researcher or research assistant to provide additional assistance in this regard. Wishing you all the best.
1 Recommendation
We have to be very careful with preserving data, particularly the experimental data. Once data is lost it is very difficult to recover it, unless doing the experiment again, which is very painful and time consuming. Usually, I store my data in different drives of my computer, as well as I keep a photocopy set.
1 Recommendation
Munira Nasiruddin Yes, this is so painful. Careful data storage at different drives is the way out.
With kind regards,
Hailemariam
1 Recommendation
Similar questions and discussions
How to interpret interaction in a glmer model in R?
Running a glmer model in R with interactions seems like a trick for me. I am new to using R.
I have a dataset called qaaf that has the following columns:
- speaker: participants in my study
- item: words as used by my participants
- realisation: the dependent variable (whether a speaker uses a CA or MA form). The target is achieved if CA is used (=1) and not so if MA (=0) is used.
- gender: independent variable (2 levels: male and female)
- age.group: independent variable (3 levels: young, middle-aged and old)
- education: independent variable (3 levels: secondary or below, university and postgraduate)
- residence: independent variable (3 levels: villager, migrant (to town) and urbanite)
- style: independent variable (2 levels: careful and casual)
- pre_sound: independent variable (3 levels: consonant, pause and vowel)
- fol_sound: independent variable (3 levels: consonant, pause and vowel)
I am testing whether my speakers use the CA form or not. This is the form of the prestigious dialect in Egypt. If they use MA, this means that they use their traditional dialect. I am trying to find out which factor (independent variable) is responsible or more responsible for using the CA form.
To run the model, I did some leveling as follows:
> attach(qaaf)
> qaaf$education =factor(education, levels=c("secondary or below", "university", "postgraduate"))
> qaaf$residence = factor(residence, levels=c('villager', 'migrant', 'urbanite'))
> qaaf$age.group = factor(age.group, levels=c('young', 'middle-aged', 'old'))
> qaaf$convergence= factor(convergence, levels=c('MA', 'CA'))
After that, I ran the model:
< Model1 <- glmer(convergence ~age.group*gender + residence*education + style + pre_sound + fol_sound + (1+residence|item) + (1+style|speaker) ,data=qaaf, family='binomial', control=glmerControl(optimizer="bobyqa", optCtrl=list(maxfun=2e5), nAGQ = 1))
The results of this model is as foillows:
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -11.227 7.168 -1.566 0.117302
age.groupmiddle-aged -25.612 9.963 -2.571 0.010148 *
age.groupold -1.970 7.614 -0.259 0.795848
gendermale -1.114 4.264 -0.261 0.793880
residencemigrant 8.056 16.077 0.501 0.616291
residenceurbanite 35.234 10.079 3.496 0.000472 ***
educationuniversity 15.985 8.374 1.909 0.056264 .
educationpostgraduate 33.529 10.573 3.171 0.001519 **
stylecasual -10.448 3.507 -2.979 0.002892 **
pre_soundpause -3.141 1.966 -1.598 0.110138
pre_soundvowel -1.661 1.540 -1.078 0.280849
fol_soundpause 10.066 4.065 2.476 0.013269 *
fol_soundvowel 5.175 1.806 2.866 0.004156 **
age.groupmiddle-aged:gendermale 27.530 11.156 2.468 0.013597 *
age.groupold:gendermale -2.210 9.928 -0.223 0.823823
residencemigrant:educationuniversity 6.967 18.144 0.384 0.700991
residenceurbanite:educationuniversity -17.109 10.114 -1.692 0.090740 .
residencemigrant:educationpostgraduate -6.901 17.836 -0.387 0.698838
residenceurbanite:educationpostgraduate -30.156 13.481 -2.237 0.025291 *
As you see, 'education' has 3 levels and 'residence' has * 3 levels = 9 levels, but there are only 4 results/estimates given in the table. The reference level in 'education' is 'secondary or below' and the reference level in 'residence' is 'villager'. Now, in interpreting the estimate of the 'educationpostgraduate: residenceurbanite' level, which is -30.156, what is the reference to which the estimate can be compared? As you see, it is significant, but significantly different from what? Plotting this interaction using the 'languageR' package (plot attached) shows that the postgraduate urbanite level uses the response/dependent variable more than any other level. If the estimate is positive. it would be easier to understand, but it is negative.
Q1: Is there a way to see the estimates of all the 9 levels in this interaction?
Q2: And how can I explain the negative estimate of the 'educationpostgraduate: residenceurbanite' level compared to the plot attached?
All the best
Related Publications
Thesis (M.S.)--Pennsylvania State University.
Thesis (M.S. 1951)--Dept. of Invertebrate Zoology and Entomology, Univ. of Utah. Bibliography: l. 54-55.
This publication is based on the case study report for the International Institute Educational Planning – United Nations Educational, Scientific and Cultural Organization (IIEP-UNESCO) research on ‘SDG4: Planning for Flexible Learning Pathways in Higher Education’ involving eight countries case studies in different regions that are developing or ha...