Survival Analysis - Science topic

Hello Seyyed Amir Yasin Ahmadi

If you are interested in this topic, you can also check out the following articles:

Thank you

In survival analysis, logistic regression is mostly used to model binary outcomes, such as whether an event has occurred by a given period. Additionally, it is modified for competing risks, models, in which many events are possible, and discrete-time survival analysis by calculating the likelihood of an event in time intervals. Furthermore, logistic regression is linked to continuous-time data by variants such as the complementary log-log model, which approximates models like Cox proportional hazards.

When you include a variable like "Number of IPOs" in the year of the event, you can use the actual number of IPOs for transactions where the event occurred. For censored observations (where the event did not occur), you can assign a value of zero or missing for the "Number of IPOs" variable, indicating that no IPO took place for those transactions during the observation period. Here are some research papers and books that might be helpful. "Credit Scoring with Macroeconomic Variables Using Survival Analysis" by Tony Bellotti & Jonathan Crook which explores the application of survival analysis to model default on a large dataset of credit card accounts, incorporating macroeconomic variables as time-varying covariates.

Another one is "Time-Varying Effects in Survival Analysis: A Novel Data-Driven Method for Drift Identification and Variable Selection" by Zakaria Babutsidze, Marco Guerzoni, and Luigi Riso. This paper discusses variable selection and stability over time in survival models with high-dimensional panel data. I have read Survival Analysis: A Self-Learning Text, 3rd Edition by David G. Kleinbaum and Mitchel Klein & An Introduction to Survival Analysis Using Stata, Revised Third Edition by Joseph H. Ibrahim which will also help you with your query.

Becareful when you are using ratio scale especially for parametric test in any inferetial data analysis. The rule is that the dependent variable must be in ratio scale and must be normally distributed. If the dependent variable is not normally distributed, then adjust the data using either log transformation or other method.

Thanks, this is a clear and very well laid out set of steps.

I do feel that for non-pharmaceutical studies that removing or minimising the placebo effect may be a disservice to the methodology that is studied.

I believe that the placebo and nocebo effects are integral to understanding treatment outcomes. Some treatments derive their efficacy from enhancing the placebo effect, which is a well-documented and beneficial phenomenon. Conversely, the nocebo effect can exacerbate perceptions of danger and elicit exaggerated responses to perceived threats. The challenge lies in isolating and accurately recording these cognitive influences.

There is a crucial intersection between science and innovation where research should focus on understanding why certain treatments yield positive results, rather than solely aiming to disprove hypotheses through traditional falsifiability methods. Both approaches—proving a treatment's efficacy and attempting to disprove it—can introduce bias.

In my own research, I have found that excluding placebo and nocebo effects might be counterproductive. Instead, we should explore ways to harness and enhance these natural healing phenomena to alleviate chronic pain. Investigating the mechanisms behind treatment efficacy can accelerate the development of effective cures more efficiently than traditional hypothesis testing.

Good.

En tant qu'enseignant chercheur en mathématiques je veux me reconvertir dans le domaine informatique spécialement dans l'intelligence artificielle

We can recreate the data If you have the analysis output. All the data recorded in the survival table

We can. If you use Stata you can make predictive model after cox regression analysis by applying generalized linear model in poisson distribution.

This can be achieved with a survival regression model (if you have a reasonable assumption about the hazard function) or Cox proportional hazards regression model (if you don't like to specify a hazard function but can assume that the hazards between the groups are proportional). There are Wald z-tests available to compare hazards between groups as for ANOVA models (which are also regression models under the hood).

Take care about error inflation by multiple testing. If you are testing a family of tests (that is: if you screen for any difference between a couple of groups), then you'll need to correct for that (-> Bonferroni, Holm).

Interview is not as straightforward as many folks think. When I have asked "why do you do this" or "tell me about this" questions, I often got responses that simply said "because that is what we do!" or their description of something I have asked about is from an insider perspective and does not address my "why" question. Essentially, I look like an adult with a beard or now white hair, but I was asking a child's question. The answer was what would be said to a child: "because!". It often took 1.5 years to get a volunteered response that was the answer to what I had been asking for 18-24 months.

As the risk set drops from 11 to 9 (ie 1 observation has left the risk set without an event), the numbers are correct:

0.598 = [1 - 1/13]. [1 - 1/12] . [1 - 1/11] . [1 - 1/9] . [1 - 1/8]

Hello Maurish Fatima,

In my experience, estimated data (e.g.: data retrieved from KM) will usually give you a "really wide" confidence interval as you say. Personally, I would have computed all three groups as three individual studies for the meta-analysis (in addition to other studies).

Cheers!

Hello Eden,

No, g*power doesn't have a direct method to estimate N for Cox regression models. However, here are some links that will get you on your way:

1. A previous RGate reply to a similar question, with links to formulae as well as a couple of online sample size calculators:

2. Guidance for degree to which a target covariate is independent of any other covariates in a model, and general sample size estimation:

3. Guidance for minimum events per variable, based on a simulation study performed on a large, real data set:

Good luck with your work.

Yes, there are methods to handle both right and left censoring in survival analysis. Censoring occurs when the event of interest (e.g., death, failure) is not observed for some individuals within the study period. Right, censoring refers to cases where the event has not yet occurred by the end of the study, while left censoring occurs when the event occurred before the study started and is only known to have happened within a certain time frame.

In the presence of right and left censoring, you can use a statistical technique called interval-censored survival analysis. This approach takes into account the time intervals within which the event of interest occurred, rather than the precise event times. Interval-censored survival analysis allows for the estimation of survival probabilities and the comparison of survival curves when the event times are only known within certain intervals.

There are several methods available for interval-censored survival analysis, including:

The choice of method depends on the specific characteristics of your data and the assumptions you are willing to make. It is important to consider the underlying distribution of the event times, the nature of censoring, and the sample size.

Implementing interval-censored survival analysis typically requires specialized software or programming packages that offer appropriate functions or procedures for this type of analysis. Consult the documentation of statistical software packages such as R, SAS, or Stata, as they often provide specific functions for interval-censored survival analysis.

Remember to carefully interpret and report the results, acknowledging the presence of censoring and the methods used to handle it in your analysis.

Many thanks, one of the authors of the paper has replied

The CONSORT (Consolidated Standards of Reporting Trials) statement provides a checklist of items to be included in reporting randomized controlled trials (RCTs) to ensure transparency and completeness. While CONSORT does not specifically address the reporting of survival analysis, it does recommend reporting the number of participants who experienced each outcome, including deaths.

To include information on deaths in a CONSORT chart, you could add a row or column to the table to report the number of participants who died in each group. For example, if you are comparing two treatment groups in survival analysis, you could add a row to the table to report the number of participants who died in each group during the study period. This information could be presented alongside the number of participants who completed the study or withdrew from the study.

It is important to report deaths and other outcomes consistently across all study arms to enable comparisons between groups. If relevant to your study, you may also want to consider including additional information, such as the cause of death or time of death.

When reporting survival analysis results, you should also report relevant summary measures, such as survival curves, hazard ratios, and confidence intervals, and provide details of the statistical methods used for the analysis.

In summary, to include information on deaths in a CONSORT chart, you can add a row or column to report the number of participants who died in each group. It is important to report deaths and other outcomes consistently across all study arms and to provide relevant summary measures and details of the statistical methods used.

Hi David. Yes, I am awaiting for a reply. Thanks.

There are classic sources that should help direct you in your quest for a unitary colonization model. Clearly, colonization cannot occur without survival (and I would say the Allee Effect is more a model of population decline and extirpation/extinction than of population colonization).

The environmental context in which a given possible colonization event is occurring is key. Island Biogeographic Theory would be informative here -- see: MacArthur & Wilson (1967) - The Theory of Island Biogeography; MacArthur (1972) - Geographical Ecology; Brown (1995) - Macroecology. Lots of graphical figures in these sources that model possible survival trajectories and outcomes of population colonizations. But, the context of any given population colonization event/process is pivotal -- this is likely a key reason you are finding it difficult to conceptualize a unitary model for population colonizations. The classic MacArthur & Wilson colonization scenario where a founding population reaches an off-shore island is just one conceptualization (e.g., there would also be need to consider whether the island was located on a continental shelf adjacent to the mainland source region for a colonizing population, or whether it was an oceanic island). Of course, the "island" conceptualization also applies to habitat blocks (e.g., protected areas) in continental regions. The taxon/taxa involved in the colonization(s) you are looking to model would be another significant variable to consider... you mention a small mammal community -- how many mammalian orders are represented? What is the body size range? Such model parameters would undoubtedly be different for mammalian taxa (as opposed to amphibian, reptile, avian, or other taxonomic groups), and could also vary considerably across different mammalian taxa (with body size being a prime limiting factor in colonization events and outcomes).

Train/Test is a method to measure the accuracy of your model. It is called Train/Test because you split the the data set into two sets: a training set and a testing set. 80% for training, and 20% for testing. You train the model using the training set.

This may help!

streg x1 x2, distribution(weibull)

I have found this paper provides a script to deal with the problem . I am still looking for a simpler solution, please share while I try to figure it out

Welcome to the jungle. Here is a web site that can help about feature selection packages

At the beginning of the study (time = 0), none of the subjects (or study participants) had experienced the event of interest you are considering in the research (that is why the survival probability is 1.0). As time continued to increase, the subjects started experiencing event of interest and the survival began to drop. Points A and B represent drop in the survival of the subjects. Point A is the drop in the survival in the 5th (year/month/ day/ hour etc) while point B is the drop in the survival in the 9th (year/month/ day/ hour etc). 10 represents the follow-up time (10 years/months/days/hours etc) for the study

subjects OR 10 (years/months/days/hours etc) is the study period. Also from the Kaplan Meier curve, the median survival time is approximately 6 years. That is the time at which 50% of the study subjects had experienced event of interest.

If survival rate did not reach 50%. Then it can't display the median. Look at the axis.

There are two, very different, ways to model an LQ model. The first assumes that the fractionated curve continues along the single dose curve. The second assumes that there is full recovery from each fraction and therefore the initial curve is repeated from the previous dose SF. The area between these curves is called the "envelope of additivity". See G.G. Steel or Peckham and Steel for more on this addition of survival curves for multifractionated doses. Interestingly, ionizing radiations (with shouldered survival curves) tend to repeat the initial portion of the curve (so-called repair of sublethal damage, but actually split-dose recovery), while some alkalizing agents, such as Bleomycin, have their curve continue along the single-dose curve (no split-dose recovery).

Your question does not make sense.run the model that you wish to run.then look at the schoenfeld residuals for lack of pattern. See the attached screenshot reference for full details. Best wishes David Booth

It sounds like you think some of your predictors may be confounded. Including multiple predictors is a way for adjusting for confounding between a predictor and the outcome. however, when the predictors themselves are confounded including multiple related predictors may not necessarily highlight THE most important ones.

as a simple example what happens if you include the same predictor twice(maybe with a little noise added in), do they get the same hazard ratio as if you only include it once?

David Eugene Booth I did not know the answer when I asked question. Then I conduct some research and find out the answer. I wrote the correct answer in the comment section, since, if someone else will be interested in such a issue, this answer will help them.

I'm not sure I comprehended your query. This is the article that may be of assistance to you. "Adelson JL, McCoach DB, Rogers HJ, Adelson JA, Sauer TM. Developing and applying the propensity score to make causal inferences: variable selection and stratification. Frontiers in psychology. 2017 Aug 17;8:1413."

Thanks David,

I ran a Kaplan-Meier: the two lines cross early in the study, and only then we can see a quite ood separation. If I understand well, perhaps it isn't surprising that the variable has a time-varying effect (increasing with time).

I also inspected the proportional hazard (PH) assumption for this variable using the cox.zph function in survival R package - the red horizontal line represents the averaged coefficient (beta) as in the Cox model (model that operates under assumption of PH), while the thin, black line is the "real" beta for the variable. The test for proportional hazard for variable X did not indicate significant departure from PH (p = 0.079, PH is violated when p < 0.05), but the added time-varying effect is significant, which is reflected by the Kaplan-Meier and the plotted coefficient over time.

Regarding the interpretation of significance of the terms in the model, I received some advise from a biostatistician: in a model where a time-varying effect is included as an interaction term (as opposed to splitting the time and calculating HR for time intervals), the main effect represents the HR where the interaction term is equal to 0 (if it's a simple X*time interaction, the inyteraction is equal 0 when time=0; it might differ in situations where there is a time transformation, for example log(time), etc.). Bottom line, even if the main effect isn't significant in the model, the (time-varying) effect of the variable is still interesting.

This has been useful to me too. Thanks Emilio

PS don't you have everything that you started with??????????

I tried it with a different PC and it worked fine. Seems like its a problem with my system.

The model should include the simple effects, as otherwise the interaction is not the difference in the effects. So the model should be like

full = coxph(Surv(Time, Censoring) ~ (age+nationality+...)*gender)

and there you would test the interaction terms via the analysis of deviance for the Cox models with and without the respective interaction term:

restricted = update(full, .~. - age:gender)

anova(restricted, full)

To check if there is a global interaction:

restricted = coxph(Surv(Time, Censoring) ~ age+nationality+...)

anova(restricted, full)

Note that it is important that the relationships between all the predictors and the log hazard rate is linear. If this is not the case, then the interaction will only catch non-linearities and this will lead to a misinterpretation.

Hi,

Any event occurring decreases the probability of the survival curve. The event is defined by the investigator. A patient improves and gets out of the treatment can be an event, likewise worsening or mortality..We can also compare across different survival curves.

Here are some notes you may find useful:

The output was generated using Stata, in case you're wondering.

The survAUC R package provides a number of ways to compare models link: https://stats.stackexchange.com/questions/181634/how-to-compare-predictive-power-of-survival-models

It sounds like your data demonstrate some feature of piecewise HR, I would suggest running analysis to accommodate this feature.

A very good reference is the book: Survival Analysis A Self-Learning Text

You can check the example on Section 6 (Starting from Page 593 to Page 597) to fit the piecewise HR Cox model, also, you can check the Stanford Heart Transplant Data on Page 265-269. The transplant may play a similar role in your data of surgery.

Fine-Gray model accounting for clusters apologies this is the one that I meant to send earlier. Best wishes, David Booth

@dorcas before you start your analysis, always make sure KM assumptions are met in your data.

This article is helpful

Best

MB

Let's see this video first:

I am not familiar with that package but you may wish to look at a similar process In our attached paper. Best wishes, David Booth

Also check https://core.ac.uk/download/pdf/79054385.pdf

Kaplan–Meier provides a method for estimating the survival curve, the log rank test provides a statistical comparison of two groups, and Cox's proportional hazards model allows additional covariates to be included. Both of the latter two methods assume that the hazard ratio comparing two groups is constant over time. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1065034/

Hi Take a look at this method. It really works well for your application.. program available from my co-author Ozgur. Best wishes, David Booth

sorry for typo '"h."

When doing a meta-analysis using survival data, you want to establish the Hazard Ratio, but you also want to give context for how that effect in the model translates into the survival of the examined patients via the risk scores that most survival models give each patient.

I would advise to report the HR for the models that you are analysing with a 95 % confidence interval for each HR and then include the p-values of the log-rank tests of the Kaplan-Meier curves. This gives the reader an idea of the effect of the features on the probability of having an event and how that effect translates then to the whole cohort.

There is this paper I found from the PLOS One journal where the authors try to do a meta-analysis of the reproting styles of differenty Phase 3 medical trials with different endpoints and methods. Maybe it could prove useful to you :

best regards, and hope this helps!

As professor David Eugene Booth said, use the plots and check the presumptions of the Cox model about your data.

You're right! I was confused, too and I could convince my collague that it would be an efficacy comparison, indeed. In the meantime, we're decided to handle it another way.

Thank you very much for your response, anyway!

I have faced this error as I forgot to convert the "survival status" variable to numeric. This error happens if we input "survival status" variable as categorical instead of numeric. Thanks for the above discussion.

It is possible. This paper covers an extension of your question:

Mathematically, nothing should stop you from doing so. Methodologically and clinically, however, I think that such a move can bring about more questions than answers in your analyses.

First and foremost, you have to justify convincingly that your uncontrolled sample and the historical comparator you are planning to benchmark it against are sufficiently similar in clinical context. However, that may even not be enough in light of unknown/unconsidered confounding.

If it is an option, perhaps a systematic review without meta-analysis, rather than with meta-analysis, may be a safer bet to synthesize the evidence.

There is route how to tackle this problem. You take all public data of merges with known internal data (stock market, profit, ...) and apply complexity measures.

The same will be done for companies without known internal data.

You will try to somehow decode the internal development from just external data using just complexity measure and statistics by comparing the above provided two classes of companies. Alternatively, you can use AI/ML as extension of statistics when it fails.

Whenever you have a common comparator, there are methods to derive a valid group estimate by indirect comparison. You may find useful guidance here:

Elvis Munyaradzi Ganyaupfu , yes we will use some ML algorithms in our analysis and we are also looking for alternative models. We use both Python and R. Our research question is on the long-term effects of hypoglycemia.

Dear Scholar

may be useful

Dear Olivia Sand . The attached article provides step-by-step instructions on how to run Cox regression analysis in SPSS and I believe it would be a perfect fit for your needs. Please check it out!

Take a look at these: they may help: file:///C:/Users/user/AppData/Local/Temp/3-31106271.pdf

R for everyone: advanced analytics and graphics | Lander, Jared P | download (b-ok.cc)

Best wishes, David

Second-by-second observation should be more than adequate to characterize hazard rate functions. You will still need to have a sufficient number of events of each type to characterize hazard rate functions with any precision, particularly as you add parameters to convey time dependence. Hopefully this is helpful to you.

The survival analysis is based on longitudinal time data. the expression of the genes should be correlated to the gene expression. The only thing I wonder about is the misunderstanding between Kaplan-Meier analysis and univariate Cox regression analysis. Are they the same? In fact, the Cox Proportional-Hazards Model is originally a multivariate statistical modeling.

For fixed categorical covariates, such as a group membership indicator, Kaplan-Meier estimates (1958) can be used to display the curves. For time-dependent covariates this method may not be adequate. But, Simon and Makuch (1984) proposed a technique that evaluates the covariate status of the individuals remaining at risk at each event time.

In Stata you can perform a simultaneous right and left censoring.

For Poisson regression:

cpoisson outcome independent1 independent2 independent3, ll(?) ul(?)

For Linear regression:

tobit outcome independent1 independent2 independent3, ll(?) ul(?)

ll --> is the lower limit for left censor (in number)

ul --> is the lower upper for left censor (in number)

For cox regression I have never used, please check:

https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwiG3fXslJbsAhWjURUIHeBQA84QFjAAegQIAxAC&url=https%3A%2F%2Fwww.stata.com%2Fmanuals13%2Fststcox.pdf&usg=AOvVaw0A-01-pTCSCWOeRVWF37l1 (page 6)

If what you are interested in is the effect (hazard ratio) for var 1 and var 2 while adjusting for others (covariates) then you are fine. You don't need every covariate to also fulfill the assumption.

Variables with time-varying effects and the Cox model: Some ... In such case, the interpretation of the models is conditional on the length of the survival time, and results should thus be ... https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-10-20

The study power depends on the number of events, the total follow-up time in person years, and the ratio between the sizes of the groups being compared.

Simulation studies have tended to the conclusion that you need ten events or more per predictor variable in your model. More recently, bigger and more comprehensive simulation studies have cast doubt on this hard-and-fast rule. Vittinghoff and McCulloch (2007), in a very widely-cited paper, concluded that “problems are fairly frequent with 2–4 events per predictor variable, uncommon with 5–9 events per predictor variable, and still observed with 10–16 events per predictor variable. Cox models appear to be slightly more susceptible than logistic. The worst instances of each problem were not severe with 5–9 events per predictor variable and usually comparable to those with 10–16 events per predictor variable.”

Since then, further simulation studies where prediction models are validated against new datasets tend to confirm that 10 events per variable is a minimum requirement (see Wynants 2015) for logistic regression. These studies are important because they are concerned with the generalisability of findings.

The second factor that will influence sample size is the nature of the study. Where the predictor variables have low prevalence and you intend running a multivariable model with several predictors, then the number of events per variable required for Cox regression is of the order of 20. As you might imagine, increasing the number of predictor variables and decreasing their prevalence both require increases in the number of events per variable.

Based on current research, the sample should have at least 5 events per predictor variable ideally 10. Sample sizes will need to be larger than this if you are performing a multivariate analysis with predictor variables that have low prevalences. In this case, you may require up to 20 events per variable, and should probably read the paper by Ogundimu et al.

The PH assumption relates to the entire model, including all predictors and covariables deemd interesting or relevant. "Restoring" PH by irgnoring or removing a covariable is not ok, as it likely demonstates that the inclusion of the covariable is relevant because it explains so much of the variance in the data that deviation from the PH assumption becomes clear.

If you have non-PH, you might start investigating if the effects of the predictors/covariables in the model are not linear. If this is not successful, an appropriate partition of the time axis might be the key (early effects are different from late effects, but the hazards are proportional within the early as well as within the late phases). If this also does not help, you might really think of stratification (what, by definition, is possible only for categorical variables). I don't consider it a good idea to categorize a continuous variable just to be able to use it for stratification. But if nothing else works, this might be a last rescue. But I would then check if the violation of the PH assumption really does more harm than the categorization of the continuous variable.