Reliability Analysis - Science topic

To get a high level of internal consistency, all of your measures need to be positively correlated. So, each of your scores would need to indicate frequently, mostly, always for strongly related behaviors. If any of them are in the "wrong" direction (i.e., negatively scored), they will be to reversed.

There are so many techniques (from qualitative to quantitative aproaches, from previsional to operational approaches) that there is no "best" technique : this stongly depends of what you want to do and what field feedback is available. This ranges from simple FMECA/HAZOP@ to stochastic Petri nets and includes Markovian and Boolean approacches (Reliability Block diagrams, Fault trees , event trees).

You could have look to my book "Reliability assessmentof safety and production systems". It provides a comprehensive description of the relevant techniques for dependability (Reliability, Availability, maintenability, safety, ...) analyses and calculations.

Perhaps you may have e free access through a subscription of your company/university.

Pandia Vadivu Pandian When you cut and paste an answer from an AI, you should state your source, just as you would in any other academic setting.

Dear Soumya Ranjan Das

First Reliability testing and then Factor Analysis. If the data is not reliable there is no point of making factors out of it.

The opinion that the first author always did the most for the results and for writing an article is not correct. Sometimes the leader of a group of authors is at the first place, sometimes all authors are given in alphabetic order.

It would be best if you first ran a principal component analysis. This will help you group your observed measures into groups or constructs. A reliability analysis follows the PCA to check if each construct (factor or group) measures what it is supposed to measure, also known as internal consistency.

Good day!

Here, there is a difference between the material failure criterion and the specific structure failure criterion. In general, the nucleation of material damage is not necessarily a condition for the failure of the structure as a whole (subcritical damage), also the structure can lose its functionality without accompanying material damage (loss of stability, change of shape, etc.). If for a specific structure and for specific conditions it is more convenient for you to use simplified criteria, for example, strain-based one, then it should still be justified quantitatively, including on the basis of generally accepted criteria for the materials fracture.

Cronbach's Alpha in SPSS Statistics - procedure, output and interpretation of the output using a relevant example | Laerd Statistics.

Dear friend Robert Bento Florentino

Sure, here are some references on using PIMS data for reliability analysis and fault prediction:

I hope you find these references helpful!

1. It usually does make sense to check the scale reliability in your sample as the reliability coefficient by definition is a variance-dependent statistic (reliability = true score variance / observed variance). Therefore, the reliability coefficients reported for a given scale in different populations may may vary, and estimated from the previous literature may not apply to your population/sample.

2. If you deal with a multidimensional scale with multiple subscales/subdomains, it does not make sense to apply, for example, Cronbach's alpha to the overall scale score (combining multiple subscales). The reason is that Cronbach's alpha is based on the model of (essential) tau-equivalence of classical test theory (CTT). Tau-equivalence implies a single-factor (unidimensional) model with equal factor loadings. Applying alpha to a multidimensional sum score is meaningless and can also be highly misleading.

3. It is not surprising that coefficient alpha would be higher for the whole scale since alpha is partly a function of the number of items (one of the fundamental laws of CTT is that all other things being equal, longer scales are more reliable because errors of measurement have more opportunity to average out). When you look at subscales, these obviously consist of fewer items and are therefore almost guaranteed to produce smaller alpha values. However, as I stated in (2), alpha would be more or less incorrect/misleading/meaningless if applied to an overall (multidimensional) scale score. Low reliability of the subscales may be due to having too few items (or unreliable items). I would only delete items if they violate the implicit unidimensionality assumption. This can be best assessed by applying confirmatory factor analysis (or IRT models) to each subscale, that is, by testing a single-factor (unidimensional) model for each subscale.

Obviously, given that factor analysis also includes the variables defined as latent and the related estimates.

Therefore, the absence or presence of weights can modify the results, even substantially.

It all depends on the purpose of your research.

It is not clear to me that all your items are in the same direction. To be certain, you should examine a correlation matrix to check whether you have only positive correlations.

Hii. I'm in the same boat with you.

I got this article which used Cronbach's alpha 0.6 . Furthermore, I completed the explanation with Dillon-alpha Goldstein's calculation that refers to Sanchez, G. (2018). PLS Path Modeling with R. www.gastonsanchez.com

What I did on the job (portraying the maintenance process from Plan to Approve to Schedule to Work to Closeout) was make a "bubble chart" with arrows from stage to stage (including skips and reversals, such as the plan was not approved and kicked back to planning) with arrows and the average time to go on the path and the number of packages to go on the path in a given time frame (such as a month).

With modern graphics, one could actually animate with ants going from mound to mound I suspect.

Steven Cooke, Thank you so much for your input!

Both answers of Steven Cooke and O.S. Abejide are correct. in addition, The reliability is the systematic calculations and prediction of the probability of limit state violation, and Vulnerability assessment is the weakest/ critical point in a system where failure is likely to start first before spreading to other members of the system

Steven Prevette that makes two of us. many thanks again for your comment. i have done a similar analysis and will post my result here once done.

Dear Pia, thank you very much for your helpful recommendation!

The R package semTools now has a new compRelSEM() function, that estimates composite reliability from estimated lavaan models. For multilevel measurement models, reliability indices defined by Lai (2021) are implemented, as well as Geldhof et al.'s (2014) less useful "hypothetical reliability" of level-specific latent components. Until version 0.5-6 is available on CRAN, the development version can be installed with syntax provided in my description here: https://github.com/simsem/semTools/issues/106

I suggest you analyze the "Tieset & Reliability analysis of a System" source code given in the below link.

Items that are measured on a 0 - 100 scale might make it even easier to assess reliability because they can (potentially) be treated as (quasi-)continuous (metrical, interval scale) variables, whereas Likert items are, strictly speaking, only ordinal in nature, often requiring special treatment in psychometric analyses.

If you have multiple items that are supposed to measure one or more factors/latent variables, the best course of action would be to run a confirmatory factor analysis (CFA) with the items as indicators of one or more latent factors to test the hypothesized factor structure first. If you find that the hypothesized factor model fits your data well/is appropriate, you can directly use the reliability estimates that are provided as part of a CFA (R-squared values for the items). In addition, composite reliability indices (reliability of the aggregate [sum or mean] of the items for a given factor) can be inferred as well from CFA. Depending on the assumptions made in the specific factor model, this may be, for example, Spearman-Brown, Cronbach's alpha, or McDonald's Omega.

The issue is not my specialty , with my best wishes

A couple of things to remember here: in a prospectus assemble the following:

a. The research questions and summarize responses by data type

b. The potential tests of the hypotheses from a

c choose tests based on a and b

d choose sample sizes based on chosen tests and type I and Ii requirements

e conduct methods of d

f answer the research questions based on e

g check the assumptions required in e

h

Prepare summary report of results

Suggestions- 0 Proper Prior Planning Prevents Poor Performance

1 get a good book on research design

2 Study it

3 Follow the suggestions for each step

4 assemble the overall plan

5 collect data as in plan 4

6see f, g and h above

7 do not ever do what you did in this

question again

Good luck David Booth

The most common way to assess reliability with a single point in time is with coefficient alpha.

Hi Mark. You should study the chapter on reliability from Julie Pallant book on how to use SPSS. It gives you practical hints on running reliability tests and eliminating the items that do not fit. Any questions? Please feel free to ask.

Yes, unfortunately it's common practice to just compute Cronbach's alpha without first testing whether the variables are essentially or strictly tau-equivalent. This may in part be because SPSS calls the procedure MODEL = ALPHA (which does not make sense in my opinion) but does not provide a test of fit for essential or strict tau-equivalence as part of the procedure (for whatever reason). When the variables are not at least essentially tau-equivalent (when they are "only" congeneric, i.e., have different loadings), Cronbach's alpha leads to an underestimate of reliability (McDonald's omega is appropriate for congeneric measures). Even worse is the (probably frequent!) case where the indicators are multidimensional (i.e., they measure more than one factor/true score). In that case, Cronbach's alpha is completely meaningless, yet you wouldn't know from SPSS output.

Essential tau-equivalence can be tested in lavaan and other SEM/CFA programs by specifying a 1-factor model with all factor loadings fixed to one and intercepts and error variances freely estimated (not set equal across variables). Strict tau-equivalence requires equal intercepts (means) across variables (otherwise same specification as essential tau equivalence).

Amjad Pervez Strictly speaking, Cronbach's alpha only makes sense when your variables are measured on an interval scale (i.e., when you have continuous/metrical/scale-level variables) and when the variables are in line with the classical test theory (CTT) model of (essential) tau equivalence (or stricter models). Essential tau-equivalence implies that the variables/items measure a single factor/common true score variable (i.e., that they are unidimensional) with equal loadings. For variables that are only congeneric (measure a single factor/dimension but have different factor loadings), Cronbach's alpha underestimates reliability. For multidimensional scales, Cronbach's alpha tends to be completely meaningless. For categorical (binary and ordinal) variables, psychometric models and scaling procedures of item response theory are usually more appropriate that procedures derived from CTT which assumes continuous (scale-level) variables.

Maybe you could describe the content of your variables (and the answer options) in a bit more detail. That would make it easier for folks on Researchgate to see which procedure may be appropriate for you.

Hello

In the non-normal situation, Spearman correlation is a suitable method

Hello Lisa,

as I suspected. These are clearly not indicator of a common underlying factor. Hence, alpha and every other internal consistency approach towards reliability are inappropriate. For its control function, however, the scale will do its job as it can be regarded as a composite of specific facets. And, yes, each of the facets won't be perfect error free indicators of their underlying attribute but that should not hurt much.

All the best,

Holger

Hi Sufyan Ghani

If this is a measurement tool such as a questionnaire or interview form, calculating a composite reliability coefficient may not be right for you. Because such a measurement tool is not standard, it cannot measure a latent trait. If it is not an objective measurement tool, it may be more accurate to deal with concepts such as inter-rater reliability.

Good luck

Your research approach is problematic. Before you ask a research question or ponder the answer to a problem, you are starting with a method and trying to fit the method to a field, not even to a specific problem. You should first ask a research question, formulate the problem, build the model and then find a suitable optimization method to solve it.

Early in my career I encountered Anne Anastasi's book "Psychological Testing". I was struck, and have remained impressed all my life, by her thought that a "questionnaire is a sample of behaviour". That is, a questionnaire is a *sample* from which the analyst makes predictions about the *ensemble* of behaviours the respondent may be said to exhibit. It makes complete sense therefore to investigate the ability of a questionnaire to allow the prediction to be made with validity (does the sample really measure the ensemble of interest?) and reliability (does the sample fluctuate at random?) Using unvalidated questionnaires is, as I used to drone to my dear students till they knew this lesson, pseudo-science and snake-oil. As in all inferential statistics, the actual *data* (in the original question, "the scores") is an accident that has just taken place. What the investigator wants to know is: what does the accident tell us ("the questionnaire")? Welcome to the wonderful world of Psychometrics!

Il faut chercher dans scimago

Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273-1296.

The below paper might be helpful:

Dear Rafel,

If you mean regression correlation, Minitab will work just fine, but it is usually done on each question response bank independently.

If you mean the "reliability" of the questionnaire, that is a different type of study requiring test groups and sampling. It SHOULD have been done already if it is a survey in use. Minitab or other statistical programs or calculations could also be used to evaluate that data. Here are two references to get you started:

Best regards,

Steven

Completely in agreement with the magnificent answer of the also magnificent researcher Dr. Pervaiz Iqbal; curiously I was going to hang something similar.

Souza, A. C. D., Alexandre, N. M. C., & Guirardello, E. D. B. (2017). Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia e Serviços de Saúde, 26, 649-659.

Not sure, but this paper could help:

Usually + or - 25%

Look up probability density function. The rebar diameter is the diameter of a steel cylander called rebar Then look the whole thing up in a civil engineering handbook to see how to use it. Best, D. Booth

I believe that there is no fixed amount for this.

What is recommended is that you exclude items in an amount so as not to fail to assess fundamental elements of your latent trait. Delete one item at a time and observe how much the correlation between the items, on average, increases (Conbrach's alpha).

I also recommend using other reliability metrics, such as McDonald's Omega

Dear Sanchit Saran Agarwal , Here is the answer

A Bayesian network, Bayes network, belief network, Bayes(ian) model or probabilistic directed acyclic graphical model is a probabilistic graphical model (a type of statistical model) that represents a set of random variables and their conditional dependencies via a directed acyclic graph (DAG). For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases.

An example of a Markov random field. Each edge represents dependency. In this example: A depends on B and D. B depends on A and D. D depends on A, B, and E. E depends on D and C. C depends on E.

In the domain of physics and probability, a Markov random field (often abbreviated as MRF), Markov network or undirected graphical model is a set of random variables having a Markov property described by an undirected graph. In other words, a random field is said to be Markov random field if it satisfies Markov properties.

A Markov network or MRF is similar to a Bayesian network in its representation of dependencies; the differences being that Bayesian networks are directed and acyclic, whereas Markov networks are undirected and may be cyclic. Thus, a Markov network can represent certain dependencies that a Bayesian network cannot (such as cyclic dependencies); on the other hand, it can't represent certain dependencies that a Bayesian network can (such as induced dependencies). The underlying graph of a Markov random field may be finite or infinite.

This would be of great help with only 13 pages http://www.cs.cmu.edu/~epxing/Class/10708-07/Slides/lecture3-BN&MRF.pdf

Dear Sanchit Saran Agarwal ,

Please have a look at these articles which talks about prior probability of warm spare gate in a dynamic Bayesian network

check this link out.

Generally, an alpha greater than 0.95 does indicate problems. I would recommend checking some aspects:

1- Is the questionnaire too long? Very long questionnaires have very high alpha. See if all items are really needed to measure the latent variable or trait.

2- Is there redundancy in my items? Are there items that have different semantics but that measure a very similar characteristic of the latent trait? Again, deleting items can help.

3- Alpha measures an average correlation between items. So it is important to check the correlation value between the items. Very high correlation items may be measuring the same characteristic as a latent trait. Checking the inflation factor of the variance can also help in the identification of multicollinearity.

I hope these tips help you.

Metrics like Cronbach's alpha and McDonald's Omega measure the consistent internal of questionnaires. But, a robust reliability measure is more naturally obtained via Factor Analysis and SEM.

When the sample is small, traditional statistical methods are not recommended. But other methods like Discourse Analysis or Content Analysis.

yes, you may proceed further and create your variable.

I agree with and Günter Becker . ANN is a prediction tool. If you have one or more equipment failure data, you can use Weibull distribution to analyze the failure.

Due to the Weibull distribution, you will be able to comment and create schedules about the behavior of the equipment and even the system.

Cost factor is the most important criterion in these scheduling. Thanks to the Weibull distribution, I suggest you take advantage of the relevant study below in order to determine maintenance periods about costs and failure.

Dear Muhammad Talha Ikram you can see the attach links

Try this link:

Do you mean someone has already reviewed it or it is done and needs to be reviewed? if it needs a review, you can send me. You're welcome.

Remove the poor loading item and rerun the reliability test

Dear Steven Cooke.

This question is related to Aviation wherein the assessment of airworthiness of some instrument found to be different in Operation manuals and Technical Manual. Operational manual is used by the Pilot/Operator whereas Technical Manual is used by Maintenance Engineering/Team. The Rating of instrument means the avionics component's operating scale with tolerance limit. For more details you can refer the avionics instrument specifications issued by various avaition agencies.

Thank you.

Absolutely right sir. @Mr.Russel King.

Thank you.

Hello Jessica,

Factor solutions can include variables that show salient affiliation with more than one factor (some personality measures are notorious for this type of structure). However, the concerns associated with cross-loading are usually: (a) the structure is more complex than Thurstone's idealized concept of "simple structure"; (b) it may make the task of characterizing what a factor represents more challenging; and (c) perhaps the variable isn't as well measured or defined as it could have been.

The answer to your question depends on your specific research aims as concerns this set of variables (and how these do or don't reflect the concerns listed above). If your goal is to derive the "cleanest" possible structure, then throwing out variables/items may be the way to go. Do recognize that the possible concern here is that you end up defining very restricted factors that may not fully represent the target constructs. As well, depending on your sample, it's possible that the resultant structure to choose may be overfitted to the sample and not generalize as well to other data sets.

In any event, if you elect to retain cross-loading items, then:

1. Yes, they are included in any representation of a factor (e.g., an estimated factor score, or just a summated score), or for score reliability estimates;

2. You present a factor structure matrix/table that shows all variables that you deem salient with each factor (factor pattern and factor inter-correlation matrices as well, if you used an oblique rotation);

3. If all loadings on a factor are negative, then you may reverse the sign and characterize the resultant variate as a reversed polarity construct estimate. (You may do the same with mixed sign loadings, as long as you reverse each variable's sign accordingly.)

4. The facetious answer is, if you get down to two items or fewer, you've likely gone too far! The more serious answer is, there is no way to predict in advance how many "keepers" there are from a preliminary set of variables that were constructed, identified, or adapted to tap one or more constructs. That's why people engage in EFA or CFA in the first place; to help identify what structures are supported by data and what structures are not.

The final note I would make is that Cronbach's alpha for an "overall" scale score might not be the best indicator, especially when you have identified multiple factors for that batch of variables. For individual factor scales, sure.

Good luck with your work.

One likely way to get that error message is if everyone gave the same answer on one of your variables. So, you need to examine the distribution of each of the variables you are using in your scale.

Dear colleagues,

I don't know if this is still of help for you, but SPSS does have a possibility to calculate the omega reliability.

There is a wonderful explanation for the installation of the extension:

written by

Prof. Andrew F. Hayes

on this page the SPSS-extension for omega calculation can be downloaded

Hope this is useful.

Best regards

It is correct you should only present evidence of validity according to your objectives that are based on the inferences of the scores obtained from your study. You should consider for the evidence of internal validity the confirmatory factor analysis.

It is apparent from your question that you are interested in observing any changes in the variable of interest after administering the intervention (participant watching the video). This is an experimental design where you are in need of a scale to measure your variable of interest.

The question on internal consistency of the questionnaire calls for a psychometric discussion, which is a different domain. If you are not evaluating the psychometric properties of your questionnaire following all the steps in scale validation, there is little point in checking Cronbach's alpha alone for the scale. Simply, obtain the achieved scores (sum of correct responses) before and after the intervention, employ paired tests.

If the variable of interest demands measurement viz validated tools, it is always advisable to use previously validated scales measuring the construct, if there are any. If there aren't any, then the question is do you want to develop one? If yes, follow the rigorous validation process.

Hi Wk,

Existing questionnaires are not perfect and are appropriate to designed situations. Surely you may improve them by modifying questions. Especially these which are problematic in terms of collective answering. I may suggest to modify questions or add more appropriate instead of removed ones.

Best

Leszek

It would be advisable to do one for each country and then a general one with multi-group invariance to evaluate the differences.

In general, no, there does not seem to be an easily accessible database of failure probabilities. Some references might be found to get started, as the following:

My own experience in a similar situation was to recognize that time moves on... START collecting that data for your operations NOW! It will be useful shortly, and sooner than you may expect. Spending too much time looking for previous solutions just prolongs the time before your own database is populated more accurately.

Oftentimes the vehicle manufacturer will have that data as well.

Hello Charles,

Can you please explain/think of your objective a bit more clearly? It is not clear whether you want to assess the accuracy of the classification method or you want to run some goodness of fit tests to understand if the model lacks something. If it is the former (as it seems from the way you framed the question), a confusion matrix might be most suitable to understand the results of your algorithm. There are several other measures to give an idea about the performance of a clustering/classification algorithm, Cohen's kappa, purity index, RAND index being some examples. Each one has its own advantage and disadvantage. I would strongly recommend you to read about them and make a decision depending on the type of dataset and problem you're working on. Hope that helps!

Thanks.

I am in a contiuous activities during this mandatory vacation but at home office..

Wish you all healthy life.

REF : https://www.clinfo.eu/itt-vs-pp/

No matter create or use Oleksandra Shatilova any measurement tool must be valid. And cronbach alpha's say nothing direct things about validity. When you handle with reverse questions, Cronbach's alpha must be rise. But it is not sufficient.

So i agree with Robert Trevethan 's advices

I suggest that you take a look to the following site :

This may be helpful.

Regards.

In order to support your explanation, I think that you must be sure that the ANOVA meets the assumptions. That gives you a guarantee about the inferences in your analysis. If the answer goes to the right way, to support your conclusions, you must calculate the power of your ANOVA. This will be an objective answer to concluded about your analysis

Thank you Sajad for sharing this interesting discussion.

I think the results of design reliability with the aid of laboratory tests such as Accelerated Life Test as well as operational reliability through Failure rates over time can be useful for designers and maintenance engineers to make a good decision about component or system.

I wonder what do you mean by the classical safety parameters ?!

Thank you Sir.