Statistical Computing - Science topic

Have any interest in a follow up to the attached paper. Best wishes David Booth

"R squared is higher" -- compared to what?

"STDEV isn't lower" -- compared to what?

The STDEV of the (absolute) differences between x- and y-values correlates with the correlation between x and y: the higher the correlation between x and y, the smaller the STDEV of their absolute differences should be. But in your case I don't see with what you are comparing your results - you have only a single sample.

Hello Shelly-Ann,

Do the computed correlations match?

Are all variables in your data set set the same way (e.g., "Scale" or "Ordinal" or "Nominal") when they are run on the two different versions of SPSS?

If so, the discrepancy may be caused by an adjustment for the set of hypothesis tests (making individual significance tests less powerful). Or, a new probability distribution function for version 28 (for t-distribution and/or F-distribution).

The way to check the first possibility is to enter just a pair of variables in the correlation dialog box and execute the analysis.

The way to check the second possibility is to evaluate some known results in both versions for the t-distribution and F-distribution.

Example:

Prob_t = compute 1 - CDF.T(2.086, 20) (should yield .0250, to 4 dec places)

Prob_F = compute 1 - CDF.F(4.35, 1, 20) (should yield .0500, to 4 dec places)

I'd also suggest you request both summary statistics and cross-product deviations and covariances (available under "Options" in the correlation/bivariate dialog box) in each version of spss and compare these. Any discrepancy would signal a potential problem.

Failing these steps, save the generated syntax for each spss version of the correlation subprogram, even if you're using dialog boxes to elicit the analyses, and make sure that these are identical across versions.

Good luck with your work.

Just do logistic regression is what I had in mind. The DV might be antcancer activity (yes /no) same for antioxidant activity. Best wishes David Booth

Respected James R Knaub Sir,

Thank you so much. Further, I will ask about some problems with it.

Yes to all your questions. The tutorial is currently under review.

Min,

Perhaps the attached paper will help. It was published in "Psychological Review," a highly prestigious and influential journal. The senior author is a well-known expert in signal detection.

Best,

Don Polzella

try fitdistrplus package in R

No I couldn't. I used Matlab instead for ANFIS. It was quite robust. You could view the architecture, rules etc quite easily. And you could also customize membership function with ease.

Read the fundamentals of the research.

Champion, D. J. (1970). Basic statistics for social research. Scranton: Chandler Publishing Company.

Thank you dear Firdos

Thank Mohammed

In my view, R is efficient for quick data analysis and visualization. It is also simple to expose results through an interactive web app with the package shiny.

However, I agree with Clément Poiret that Python is almost inevitable when you work in industry thanks to its versatility and the ability to respect processes of continuous integration. I've first learned coding with R at school, few years after I had to use Java and C++. This lead me to review and improve the way I coded with R. After that I've learned Python for OOP and statistical programming, this lead me to improve again my R skills. Now with few years of experience, I often switch between R and Python depending on the task. As I said firstly, I always use R and RStudio for data exploration (I found it powerful to deal with big datasets and parallel processing) and quick reports with Rmarkdown (HTML and TeX). I necessarily use Python when I need robustness for heavy project and OOP (e.g. to improve factorization and inheritance).

Finally, I would say that there ALWAYS be a rigorous way to code in R, but it is not mandatory and the language do authorize bad practice to obtain same results. It is not just about code performance, but also about the syntax of the functions or even simply about the code implementation. For statisticians working in teams who learned only R, I would recommend to use packages with documentation (e.g. roxygen2) and unit tests (e.g. testthat).

The function stats::rstudent() extracts studentized residuals.

Dear Hapuarachchige Don Nelaka Shayamal Priyankara

You can use the IDS in a noncentralized process also.

Regards

with

The lack of standard error depends on your software, and even then it only applies to the variance terms. The reason for this is the variance cannot go negative and the sampling distribution can often be expected not to be asymptotically normal but skewed. So just explain this in you results table.

The Fokker-Plank equation can be treated as a so-called forward Kolmogorov equation for a certain diffusion process.

To derive a stochastic equation for this diffusion process it is very useful if you know a generator of this process. Finally, to find out a form of the generator you have to consider a PDE, dual to the Fokker-Plank equation which is called the backward Kolmogorov equation. The elliptic operator in the backward Kolmogorov equation coincides with the generator of the required disffusion process. Let me give you an example.

Assume that you consider the Cauchy problem for the Fokker-Plank type equation

u_t=Lu, u(0,x)=u_0(x),

where Lu(t,x)=[A^2(x)u(t,x)]_{xx}-[a(x)u(t,x)]_x.

The dual equation is h_t+L^*h=0, where L^*h= A^2(x)h_{xx}+a(x)h_x.

As a result the required diffusion process x(t) satisfies the SDE

dx(t)=a(x(t))dt+A(x(t))dw(t), x(0)= \xi,

where w(t) is a Wiener process and \xi is a random variable independent on w(t) with the distribution density u_0(x).

You may see the book Bogachev V.I., Krylov N.V., Röckner M., Shaposhnikov S.V. "Fokker-Planck-Kolmogorov equations"

Would you like to provide R code for IPTW?

Dear Leopoldino

I guess that you are referring to OR (Operational Research) and you use the Brasilian OP (Pesquisa Operacional). If it is so, I believe that the most important aspect of your remark is that OR, and most especially Linear Programming (LP) solves real problems, which is possible because its algebraic structure of inequalities. It allows the construction of a scenario model by far much more representative than in any MCDM methods.

LP was developed in 1939 by Leonid Kantarovich, and for this development he was awarded in 1952 the Nobel Prize in Economy. It is then the granddad of all present-day methods for MCDM.

The actual algorithm, the Simplex, is due to the genius of George Dantzing which developed it in 1948. This same algorithm is still used to day, after 70 years, and is, according to some sources, used by about 70,000 large companies world-wide.

LP processes large amounts of information, it is irrelevant the number of alternatives and criteria, which can be in the thousands each, and it is so important, that Excel incorporated it as an add-in since 1991.

However, LP has two severe drawbacks; one of them is that it works with only one objective and with only quantitative criteria, which is not very realistic in nowadays projects. Nowadays, these two problems have been superseded by new methods and software based on LP, which do not yield an optimal solution, as the original Simplex, but one which is probably very close to it.

What is important is that LP allows for modelling very complex scenarios, incorporating features than none of the more than two dozen MCDM in the market can handle, and for this reason, it is able to model real problems, by establishing restrictions, dependencies and even correlation.

In addition, it does not produce Rank Reversal.

I have been using LP for decades and in the late 70s was fortunate enough to act as a counterpart of the Massachusetts Institute of Technology (MIT), to solve by LP in a two years project, a very complex problem related with a river basin.

Since then, I solved more than one hundred problems in very different areas, and many of them have been published in my books, and at present, I am trying to promote its use in RG, where, as you properly say, rationality is paramount.

LP mathematics is a little complex, however, a user does not need to know it, in the same way as he does not need to know the mathematics of AHP, PROMETHHE or TOPSIS. As matter of fact, LP is easier to use than other methods since no weights are needed.

A couple of weeks ago I proposed in RG to develop a course on LP, however, nobody was interested.

I found the answer in this paper.

I am with Alexander & James on this. First, Nearly, all so-called big-data is "it-is-what-it-is" data (i.e. The analyst has no control over how the data is collected. Under these circumstances the selection process for the sample is non-random & the selection probabilities are typically unknowable. One is often not even certain if the sample is from 1 unique population.) Under such circumstances biases can arise & their effects are not measurable. For something like sales; big-data can be highly applicable & population specific because it represents a sample from one's particular population: your customers. Yet, if it represents health records, one has to wonder whether or not the source for these record misses some parts of the population or over-represents sub-populations. I think solid "statistical thinking" can help big-data analysis more than the other way around.

1. You can use a structural equation approach to meta-analysis, as suggested by Gordon. I recommend another book by Mike Cheung: https://www.amazon.de/Meta-Analysis-Structural-Equation-Modeling-Approach/dp/1119993431. For this, you would use the individual correlation matrices of the primary studies to generate a meta-analytical variance-covariance matrix and run SEM on that. This method is preferable to first meta-analysing the bilateral relation and using the matrix of those relations as input to SEM.

2. You can use the output of SEM as you would treat any regression method for inclusion in meta-analysis. Hence, you can use the regression coefficients (or its derivative: partial correlation) and run the meta-analysis on that. Check the first question here: http://www.erim.eur.nl/research-facilities/meta-essentials/frequently-asked-questions/ (disclaimer: I wrote the answer). This is based on the work of Ariel Aloe and others: DOIs: 10.1080/00221309.2013.853021 and 10.3102/1076998610396901

Good luck!

Robert

PS: I think Martin Bjørn Stausholm misunderstood your question as referring to standard errors. His answer is absolutely correct (I think) but doesn't relate to structural equation modelling (SEM)

ıt means that all respondents answer your one of the items same answer thus there is no variability in that item for example all respondents answered The item 1 as 3 check item descriptive statistics you will see at least one item has zero standart deviation

Entropy: A concept that is not a physical quantity

Putting it simply, When your trend analysis give you a significant trend (positive or negative) Sen's slope is than to capture the magnitude of that trend. Say your MK test revealed that temperature increased yearly between 1950-2000, here Sen's slope will tell you on average how much temperature has changed each year.

Hope it helps you

Try tetrachoric correlation coefficients. psych package implemented it.

If you have no variance in these items you will run in a number of problems (like missing correlations where they are theoretically expected). When you aim to use your questionnaire in a group similar to the one you got your test-retest-data from you should consider excluding these items. I would keep them only if they cover something very important that is not assessed by any other item and if there is at least the possibility that a subject has a different answer than all others.

In my experience, this happens when you don't have variation in responses during either the test or retest.

For example, if you were calculating kappa for a yes/no question, and everyone answered yes during the test but a few reported no during the retest, you would not be able to calculate kappa.

You would be able to report % agreement and note that you can't report kappa because test or retest data are a constant.

Dear Basith,

In one of my papers (see at the end of this response) I clarified that

"Trend direction can be significant for a very small magnitude of increase or decrease which may be unimportant in practice. On the contrary, Trend direction may be insignificant for change whose magnitude must not be ignored in environmental management decision-making".

To reliably interpret the results of the trend analysis, you need answers to the following questions:

For answers to the above questions as well as useful information on how to interpret your trend results, you may find the following papers may be so relevant for your case. 

best wishes

Can you give a simple, reproducible example?

If you do not post a reproducible example  (i.e, part of the data) it is hard to help. In general I discourage to use nested loops in R.

Dear Timothy

it is only about illustrating the correlation so the extreme values should be of minor importance (anyway, in case of extreme values, correlation is not really appropriate);

for other purpose, rerandomisation might be better than Bootstrap even though I think that both methods are relatively poor with rare events as they usually are not represented in the original sample so that they will not show up neither in the Bootstrap distribution nor in the rerandomisation.

Hiba, you should use R to do either the Bootstrap or rerandomisation.

Dear Vasana,

As far as I know this is a problem with no general solution working for any situation. Given multivariate observations, I realize that you are asking for a GOF test for the marginal distribution, do you? There are several results for normality in the time-series framework. In general, to construct a GOF test , it is necessary to specify the marginal distribution (Gamma, Weibull, Poisson, Binomial...) and the correlational model (AR(1), ARMA(p,q), ...) as well. For instance, in the following paper you can find and extension of the classical Fisher dispersion test for the Poisson distribution with an AR(1) structure (INAR(1) model):

Schweer S, Weiß CH (2014) Compound Poisson INAR(1) processes: stochastic properties and testing for overdispersion. Comput Stat Data Anal 77:267–284

Best regards.

Standard statistics.  Depends on what degree of variability you can tolerate.  The standard error is proportional to 1/variance.  So for 10% standard error (i.e. Within 10% of the true value) will require 100 experiments.  For 1%, then 10000 measurements.  People never like the conclusions, but the math is simple.

Stephen Politzer-Ahles has it right.

There is a subtle difference between a confidence interval and its associated random interval. For example, a 95% confidence interval for the mean is one instance of an associated random interval.  The random interval has a 95% of containing the mean, not the confidence interval.

Both the mean and a confidence interval are fixed.  They do not vary. Either the confidence interval contains the mean or it does not.  There is no probability that can be logically assigned to it.  On the other hand, there can be a probability assigned to a random interval containing the mean.

As explained below, I believe that the issue comes down to understanding the difference between the "mean" \mu and the "sample mean" \bar{X}.  The sample mean is a random variable, while the mean is a fixed number.

For a known variance \sigma^2, and a normally distributed random variable X, the random interval associated with a confidence interval for the mean is [\bar{X} - z\sigma/sqrt{n},\bar{X} + z\sigma/sqrt{n}], where z is a fixed number that depends on the confidence level (e.g. 95%) and n is the sample size. This interval is random because it contains the random variable \bar{X}.  Once you assign an instance of \bar{X}, the interval is no longer random.  It either contains \mu or it does not.

For teaching confidence intervals in an introductory statistics class, one might consider the following outline.

(1) Explain what a random variable is, (2) Explain what a random sample is, including what IID means, (3) Define \bar{X} in terms of a random sample (4) Explain the difference between \mu and \bar{X} (\bar{X} estimates \mu, \bar{X} is random, \mu is not). (5) Explain what a confidence interval is (for the mean).

There are of course several steps in between to explain, but these are essential from my perspective.  It may be that doing something less will lead to the sorts of confusion already mentioned.  On the other hand, it may be too much for an introductory class, depending on the ability of the students involved.  Ultimately, it's a judgement call.  

By something less, I mean the approach taken in the following reference:

Orkin, M. and Drogin R., "Vital Statistics", McGraw-Hill, 1975.

The approach I've suggested above is more in line with the following:

Meyer, Paul L., "Introductory Probability and Statistical Applications", Addison-Wesley, 1970.

Both are introductory, but assume different levels of mathematical experience and/or ability; the latter being at a higher level.

Hi Jhonnhy.

You can fit linear models with correlation structures for the error using package nlme (https://cran.r-project.org/web/packages/nlme/nlme.pdf). There is an argument `correlation` in the `lme` function to model spatial correlation. Also the function `Variogram` is used to compute the semi-variogram. Argument `form = ~ x + y` represents a two-dimensional position vector with coordinates x and y, which I think is your case.

The error that you are facing is related to the values of your variable not fittin g the requirements of the gamma function. may be you need to change the method and function you are using for the analysis. or may be make transformation that fits the data to the gamma function.  

ICA is usually applied in ECG signal processing for denoising, artifacts removal and source separation (e.g. fetal ECG, in which it can be used to separate the mother's signal from the fetus).

You can find more info on the book Advances in Electrocardiograms - Methods and Analysis, Chapter 19: Independent Component Analysis in ECG Signal Processing

to do power analysis to estimate your sample size, you have to write your hypothesis, and based on that you decide what statistical test you will use. It should be one of the inferential statistics. so you need to determine the following: alpha {standard to be .05}, power [standard to be .80], effect size {small, moderate, or large, each test has its own value, you can find these values in the net}. Then download free programs to calculate the sample size such as G. power.

   

Dear Bruce,

Thank you for your useful suggestions! I'll try on that and see if it's suitable.

Logistic regression analysis is commonly used when the outcome is categorical. By using the natural log of the odds of the outcome as the dependent variable, we usually examine the odds of an outcome occurring or not, and the relationships are linearized similar to the multiple linear regression (Hosmer, 1989).When the outcome is not dichotomous; we should distinguish the outcome type first, which is often ignored by some investigators. There are two kinds of logistic modeling: multinomial logistic regression handles the case of a multi-way categorical dependent variable (Friedman, 2010) and ordered logistic regression handles ordinal dependent variables (Simon, 2014). The predictor could both continuous or categorical for the two types of logistic regression.  Using nominal, k level outcome, has k -1 dependent variables, hence k-1 incerception and k-1 parameters each continuous predictor, and (k-1)*(m-1) for each m-level categorical predictor. Treat categorical predictor as ordinal by assign different reasonal scores, we use one parameter to replace the m-1 parameters to increase testing power.

 Multinomial logistic model

The data set of the following example contains the results of a hypothetical user testing three brands of computer games. Users rated the games on a five-point scale from very good (vg) to very bad (vb). The analysis is performed to estimate the differences in the ratings of the three games. The variable score contains the rating of scales, and the variable game contains the games tested. The variable count contains the number of testers rating each game in each rating group. The baseline logistic is used to define the regression functions: Using the one level as reference then LOG(p1is/pijm) as outcome function. See following SAS program

There are 4 outcome functions the log odds with score of very bad (reference group):

ln(p(vg)/p(vb)), ln(p(g)/p(vb)), ln(p(m)/p(vb)), ln(p(b)/p(vb)). The SAS coding is:

Data Compgame;

input count game$ score$ @@;

datalines;

70 game1 vg 71 game1 g 151 game1 m 60 game1 b 46 game1 vb

20 game2 vg 36 game2 g 130 game2 m 80 game2 b 70 game2 vb

50 game3 vg 55 game3 g 140 game3 m 72 game3 b 50 game3 vb

;

Proc logistic  data=Compgame  rorder=data; /*rorder function kee the outcome functions as the order as the order in data: vg,g,mb,vb*/

freq count;

class game /param=glm;

model score = game /link=glogit;

run;

Table 10 summarizes the model fitting and estimating results.

Table 10: Multinomial logistic regression*

Parameter

score

Estimate

Standard error

P value

Model Fitting

Intercept

vg

0.000

0.200

1.000

AIC

3356

3323

Intercept

g

0.095

0.195

0.626

SC

3376

3383

Intercept

m

1.030

0.165

<.0001

-2 Log L

3348

3299

Intercept

b

0.365

0.184

0.048

OR*

95% CI

game1

vg

0.420

0.276

0.128

1.522

0.886

2.612

game1

g

0.339

0.272

0.213

1.403

0.823

2.391

game1

m

0.159

0.236

0.500

1.172

0.739

1.86

game1

b

-0.099

0.269

0.713

0.906

0.535

1.534

game2

vg

-1.253

0.323

0.000

0.286

0.152

0.538

game2

g

-0.760

0.283

0.007

0.468

0.268

0.815

game2

m

-0.411

0.222

0.064

0.663

0.43

1.024

game2

b

-0.182

0.245

0.457

0.833

0.515

1.347

·         The Reference group is game 3 and score=vb

Reading results:

1.      There are 4 intercepts for the four baseline logit outcomes, with the first being for the first logit: ln(p(vg)/p(vb)), which is -1.25E-13=0. It is the log odds for Game 3 (see the raw data: for Game 3, vg=vb=50, hence ln(p1/p5)=0.  Similar to the intercept1, all intercepts are the log odds with p5 for GAME 3.

2.      There are 8 regression coefficients β, which are the differences of the four logits between the Game 1 or Game 2, with Game 3 respectively. The first β of Game 1 is vg =0.42, which is logit difference between Game 1 and Game 3 for the response function ln(p(vg)/p(vb)). It is positive, meaning that the evaluation of Game 1 is better than that of Game 3, with more evaluations of “very good” received. And exp(0.4199)=1.522 is the odds ratio between GAME 1 and GAME 3 for comparing the very good score with very bad score. The percentage of the stating (?) GAME 1 to be very good vs very bad, is roughly 52% higher than GAME3. However the difference is not significant and the p-value is only about 0.13.

3.      The interpretation is similar for others, for GAME 2, the percentage ratio receiving scores vg versus vb is significantly lower than for GAME 3, with the odds ratio 0.286.  The very good rating for Game 2 is much lower comparing with Game 3.

If we only use score=vg and score=vb, it is a binary outcome, use the logistic we get the same results for the multinomial logistic regression for the response function log(p(vg)/(1-p(vg))=log(p(vg)/p(vb)). The parameter estimate for Game 1 is 0.4199 and the odds ratio vg to vb is 1.52. Hence, for the data above, the multinomial model is just combining the dichotomous logistic regressions together. However, if there are other cofactors in the model the model will be more complex. If the model has different intercepts and different slopes, the combined multinomial logistic model and the binary logistic model will be the same.

Ordered logistic model:

Since the score measurement is clearly ordered, assuming p1, p2, p3, p4, p5 be the probabilities to be testes as vg, g, m, b and vb, the cumulative logistic functions defined as follows and to be defined as proportional:

Response function                              Assigned score

Very good: ln(p1/( p2+ p3+ p4+ p5)),    0

        good: ln(p1+ p2)/(p3+ p4+ p5)),     1

   medium: ln(p1+ p2 + p3)/(p4+ p5)),    2

          bad: ln(p1+ p2 + p3+ p4)/ p5),      3

The regression equation is

     2.5.2

The SAS coding is

proc logistic  data=Compgame  rorder=data; /* rorder function assigns the order of the four outcome functions as the order of the data, the scores are 0, 1, 2 and 3*/

freq count;

class game /param=glm;

model score = game/link=clogit ; /*clogit performing the ordinal logistic regression, assuming the increasing rate or the four lgits is constant*/

run;

The parameter estimations are list in Table 11:

Table 11: Analysis of Maximum Likelihood Estimates

Parameter

 

DF

Estimate

Standard

Error

Wald

Chi-Square

Pr > ChiSq

Intercept

vg

1

-1.9087

0.1189

257.5924

<.0001

Intercept

g

1

-0.9356

0.1022

83.7943

<.0001

Intercept

m

1

0.7305

0.1004

52.9434

<.0001

Intercept

b

1

1.8493

0.1162

253.4371

<.0001

game

game1

1

0.3098

0.1307

5.6179

0.0178

game

game2

1

-0.5748

0.1365

17.7412

<.0001

game

game3

0

0

.

.

 

 

It shows that assuming the odds to be proportional, the clogit uses the cumulative odds. the game effects on the assigned score, the four cumulative logits is assumed to have the same ‘distance’, ranged from 0 to 3 .  The slope estimation for factor of game is relative change of the logarithm of cumulative odds the Game3. For Game1, the slope is 0.3098, it implies the counts are non-even comaring with Game3, the logarithm of cumulative odds ratio is 0.3098, which is the change from one category to another category in sequential with the reference levle of vb. Hence there is more good reading for Game1 comaring with Game3. For Game2, the slope is -0.5748, hence there are more bad reading for Game2 comparing with Game3.

I had the same problem - I had to make sure that the factor was fixed and that seemed to solve it! 

try SAS 9.1 as recommended in one article (Analysis of 2 3 2 tables of frequencies:

matching test to experimental design International Journal of Epidemiology 2008;37:1430–1435

doi:10.1093/ije/dyn162  Published by Oxford University Press on behalf of the International Epidemiological Association

The Author 2008; all rights reserved. Advance Access publication 18 August 2008) i read early. by JOHN LUDBROOK

R

Function "breakpoints" in package strucchange is based on piecewise linear models. It uses dynamic programming to find breakpoints that minimize residual sum of squares (RSS) of a linear model with m + 1 segments. The Bayesian Information criterion (BIC) is used to find an optimal model as compromise between RSS and number of parameters.

The linear model can be quite general and may include covariates. A parameter "h" can be set to determine minimal segment size. This is especially important for relatively small samples.

More can be found:

- in the very well readable book: Kleiber, C. & Zeileis, A. Applied Econometrics with R Springer, 2008,

- in the package vignette of strucchange: http://cran.r-project.org/web/packages/strucchange/vignettes/strucchange-intro.pdf

- in several papers of Prof. Achim Zeileis and

- in additional material found at the homepage of the author: http://eeecon.uibk.ac.at/~zeileis/

for example: http://eeecon.uibk.ac.at/~zeileis/teaching/AER/Ch-TimeSeries.pdf#58

It's a really smart and powerful technique -- I like it :)

Thomas

Thanks Ellen Hertzmark, I have 642  observations with only 3 variables. I will try the steps you have mentioned and see what happens. Funny enough, if I leave out a semicolon it  quickly gives me an error. I was hoping the output will be that quick .

We need more information on your data and experimental design to give a meaningful answer. Either post information about the design (and how you measure hatchability) or email me, and I will respond. Good luck :)

I have required the "standard deviation" (strictly the rms) of the parameters I am evaluating to be less than 20% fir a single "run."

You basically need 2 arguments, one being the model you calibrate with some data, and other data you want to predict their associated value

predict(name of your previously run model, the new data you want to predict)

this site may also help you:

You can use the spatstat package (http://www.spatstat.org/spatstat/) which offers the possibility to fit Ripley's K function. Spatstat also seems to work quite well also for large data sets. You might have a look at the function Kest().

see the pracma package and the note at

but also package neldermead may have one implementation.

Another option might be to use a generalized Poisson distribution.  I know this can be done in STATA, but I don't know if/where one would do it in SAS.

Or, you could transform your data monotonically - say, via square root or log function.  This would be workable, since your minimum is >0 on the original scale.

Usually, you would take a random number generator (RNG) that can provide uniformly distributed values between 0 and 1. Here's how the mapping is generally done: First, from your probability distribution function (PDF) you generate the cumulative distribution function (CDF) through integration over x. The CDF will always be invertible. Now, you can pick any random number from a uniform distribution and look up the x-value of your function through the inverse CDF. This x-value will be a random number from your PDF. Sometimes the CDF cannot be determined by a formula, so that you'll have to find a numerical approach to this. With the right keywords you should be able to find papers on this.

Here's a link on what Wikipedia has to say on this:

1872 Orange pog.svg – Ludwig Boltzmann presents his H-theorem, and with it the formula Σpi log pi for the entropy of a single gas particle.

1878 Orange pog.svg – J. Willard Gibbs defines the Gibbs entropy: the probabilities in the entropy formula are now taken as probabilities of the state of the whole system.

1924 Red pog.svg – Harry Nyquist discusses quantifying "intelligence" and the speed at which it can be transmitted by a communication system.

1927 Orange pog.svg – John von Neumann defines the von Neumann entropy, extending the Gibbs entropy to quantum mechanics.

1928 Red pog.svg – Ralph Hartley introduces Hartley information as the logarithm of the number of possible messages, with information being communicated when the receiver can distinguish one sequence of symbols from any other (regardless of any associated meaning).

1929 Orange pog.svg – Leó Szilárd analyses Maxwell's Demon, showing how a Szilard engine can sometimes transform information into the extraction of useful work.

1940 Red pog.svg – Alan Turing introduces the deciban as a measure of information inferred about the German Enigma machine cypher settings by the Banburismus process.

1944 Red pog.svg – Claude Shannon's theory of information is substantially complete.

1947 Purple pog.svg – Richard W. Hamming invents Hamming codes for error detection and correction. For patent reasons, the result is not published until 1950.

1948 Red pog.svg – Claude E. Shannon publishes A Mathematical Theory of Communication

1949 Red pog.svg – Claude E. Shannon publishes Communication in the Presence of Noise – Nyquist–Shannon sampling theorem and Shannon–Hartley law

1949 Red pog.svg – Claude E. Shannon's Communication Theory of Secrecy Systems is declassified

1949 Green pog.svg – Robert M. Fano publishes Transmission of Information. M.I.T. Press, Cambridge, Mass. – Shannon–Fano coding

1949 Green pog.svg – Leon G. Kraft discovers Kraft's inequality, which shows the limits of prefix codes

1949 Purple pog.svg – Marcel J. E. Golay introduces Golay codes for forward error correction

1951 Red pog.svg – Solomon Kullback and Richard Leibler introduce the Kullback–Leibler divergence

1951 Green pog.svg – David A. Huffman invents Huffman encoding, a method of finding optimal prefix codes for lossless data compression

1953 Green pog.svg – August Albert Sardinas and George W. Patterson devise the Sardinas–Patterson algorithm, a procedure to decide whether a given variable-length code is uniquely decodable

1954 Purple pog.svg – Irving S. Reed and David E. Muller propose Reed–Muller codes

1955 Purple pog.svg – Peter Elias introduces convolutional codes

1957 Purple pog.svg – Eugene Prange first discusses cyclic codes

1959 Purple pog.svg – Alexis Hocquenghem, and independently the next year Raj Chandra Bose and Dwijendra Kumar Ray-Chaudhuri, discover BCH codes

1960 Purple pog.svg – Irving S. Reed and Gustave Solomon propose Reed–Solomon codes

1962 Purple pog.svg – Robert G. Gallager proposes Low-density parity-check codes; they are unused for 30 years due to technical limitations.

1965 Purple pog.svg – Dave Forney discusses concatenated codes.

1967 Purple pog.svg – Andrew Viterbi reveals the Viterbi algorithm, making decoding of convolutional codes practicable.

1968 Purple pog.svg – Elwyn Berlekamp invents the Berlekamp–Massey algorithm; its application to decoding BCH and Reed-Solomon codes is pointed out by James L. Massey the following year.

1968 Red pog.svg – Chris Wallace and David M. Boulton publish the first of many papers on Minimum Message Length (MML) statistical and inductive inference

1970 Purple pog.svg – Valerii Denisovich Goppa introduces Goppa codes

1972 Purple pog.svg – J. Justesen proposes Justesen codes, an improvement of Reed-Solomon codes

1973 Red pog.svg – David Slepian and Jack Wolf discover and prove the Slepian–Wolf coding limits for distributed source coding.[1]

1974 Red pog.svg – George H. Walther and Harold F. O'Neil, Jr., conduct first empirical study of satisfaction factors in the user-computer interface[2]

1976 Purple pog.svg – Gottfried Ungerboeck gives the first paper on trellis modulation; a more detailed exposition in 1982 leads to a raising of analogue modem POTS speeds from 9.6 kbit/s to 33.6 kbit/s.

1976 Green pog.svg – R. Pasco and Jorma J. Rissanen develop effective arithmetic coding techniques.

1977 Green pog.svg – Abraham Lempel and Jacob Ziv develop Lempel–Ziv compression (LZ77)

1989 Green pog.svg – Phil Katz publishes the .zip format including DEFLATE (LZ77 + Huffman coding); later to become the most widely used archive container and most widely used lossless compression algorithm

1993 Purple pog.svg – Claude Berrou, Alain Glavieux and Punya Thitimajshima introduce Turbo codes

1994 Green pog.svg – Michael Burrows and David Wheeler publish the Burrows–Wheeler transform, later to find use in bzip2

1995 Orange pog.svg – Benjamin Schumacher coins the term qubit and proves the quantum noiseless coding theorem

2001 Green pog.svg – Dr. Sam Kwong and Yu Fan Ho proposed Statistical Lempel Ziv

2008 Purple pog.svg – Erdal Arıkan introduced Polar Codes, the first practical construction of codes that achieves capacity for a wide array of channels.

Hi Peter,

Exactly the same as in ENVI!

Have apeek at Envi Freelook and inspire yourself.

Cheers,

Frank

You can calculate R2: 1-rss/tss.

The residual sum of squares (rss) can be calculated using ctree's "predict" function, so if "model_ctree" is your ctree model and "y" is your dependent variable:

rss <- sum((y - predict(model_ctree))^2)

tss <- sum((y-mean(y))^2)

r2 <- 1-rss/tss

Note that this R2 is a prediction on the learning data. If you want to predict on new data, you will have to specify the test set conaining the new data using the "newdata" parameter in the "predict" function. If your test set is "testset", use:

predict(model_ctree, newdata = testset)

If you do not have fresh data for the test set, you can hold out an amout of cases from your entire sample by randomly splitting it up into a learning sample (that you use to fit the ctree model) and a test sample (that you use for prediction).

Characters?? or variables? question requires elaboration..

Evaluating the security of a random sequence for cryptography (I assume that what you mean by the security), please check out the NIST SP 800-22 (http://csrc.nist.gov/publications/nistpubs/800-22-rev1a/SP800-22rev1a.pdf)

That's the reason why Becker-Döring cluster theory and its modifications (coagulation-fragmentation equations) is still quite popular amongst mathematicians and physicists. One investigates e.g. the first-order phase transition "gas -> liquid" via the formation of larger and larger clusters, formed out of a monomer bath (i.e. the gaseous phase). Here a cluster consists of several monomers sticking together. Once a critical clustersize is exceeded one identifies these "large" clusters with the occurance of droplets (i.e. the liquid phase).

The stochastic processes of cluster formation are usually described by master equations. For Markovian stochastic processes these master equations are represented by infinite systems of ordinary differential equations of 1st order in the time parameter t. And as you observed correctly, under certain conditions one recovers a time scale tending to zero, i.e. metastable states. Furthermore, as you also mentioned, the physical systems are far from equilibrium. Hence Onsager's reciprocity relations approach is not applicable. Oliver Penrose, a former post-doc of Lars Onsager, has dedicated quite some time researching the issues you mentioned in the 70s, 80s and 90s, also Kurt Binder, Joel Lebowitz and Enzo Olivieri to name but a few pioneers.

A drawback of this mathematical picture, however, is the need of a micro-physical theory for the transition rates. If one really had a serious theory of the transition rates for water and the "liquid -> solid" phase transition ("freezing"), say, then one could investigate a possible theoretical explanation for the Mpemba effect.

Here are a few thoughts; see if they help.

1. This is the classic bias-variance trade-off problem: Too few bins result into too much bias (i.e., deviation from the true but unknown underlying density) but low variability (of the histogram) across different data realizations (from the same data-generating process), whereas too many bins lead to little bias but too much variability. The basis for dealing with such bias-variance trade-off problems is usually a squared loss between the histogram estimator and the true but unknown density, and the associated risk function (i.e., expected loss). All optimal binsize prescriptions (such as Freedman-Diaconis) attempt to balance the bias of a histogram with its variance. This standard formalism can be found in, e.g., Larry Wasserman's All of Statistics and All of Nonparametric Statistics (both Springer).

If you use an optimal binsize prescription, and the data meets the assumptions that the prescription is based on, then information loss is autoatically minimal according to that prescription.

2. Also remember that a histogram is just one estimator for the probability density of your data. That too a discrete one. If the underlying density is known to be continuous, then other density estimators (e.g., kernel density estimators) may be more appropriate.

3. The CDF need not be estimated from the histogram. There is, e.g., a nonparametric estimator called the empirical distribution function (EDF) that estimates the CDF directly from the data. The EDF does not depend on any adjustable parameter such as the bin size, and its formal properties are well-understood.

Assuming you're not interested in the more high-end approaches like random forests or model averaging, you might want to try a mixed effects model where you include database as a random effect.