Data Model - Science topic

only take the difference of that variables which are non stationary.

Thank you J.C !

Do you have an active link for downloading the model by any chance?

Data about customers are sensitive and confidential and I doubt if any bank will like to release data about customers' loans. In my country (Nigeria) there may not be possibility of getting online information to capture this data because of fraudsters. I therefore have no idea of how clearance and permission can be given by Micro finance bank to get customers data about loans for research..

Daniele Sasso

I agree here is not a consistent basis for judging a contribution. I think this is because there is a wide range of opinions - many of which are oriented not to scientific advance but to publishing with little contribution ("publish or perish"). From this comes my criteria for spending time on an article. I defend my criteria of abstract length by noting long abstract reflect a lack of clear thinking. The article can only be vague also. Perhaps, this is too harsh but I've spent a lot of wasted time before I followed the old APS criteria.

You last point is why I asked the question to myself and on RG. Exactly correct, the historical judgement is the only criteria but then it's for the future - not the present.

The idea that the valid model will prevail depends heavily on it being recognized. Part of my quest is to gain other's views for contribution into my thinking/model and, perhaps, modification of my thinking.

following this link

You can try Mixed model (random for periods and fixed for cross-sections or vice versa), if that didn't succeed, you should take random(for period only) and Non for the cross section, or reverse. you can do this by fixed also, after that you should check your choice by Housman or Chow test(redundant test in EVEWS) .

Akhilesh Nautiyal

1. Use the downhill simplex method to minimize a function.

2. Use the BFGS algorithm to minimize a function.

3. Use the nonlinear conjugate gradient technique to minimize a function.

4. Using the Newton-CG approach, minimize the function f.

5. Use a modified Powell's approach to minimize a function.

Brown Chitekwere,

May I ask if you find your answer to share it here, please?

In addition, when I ran the model, I received the error "rbind error: "names do not match previous names." Do you have any idea about it?

I appreciate your help.

Kind regards,

Mona

Jochen Wilhelm I agree very much with your statement that "you should better do more research on the meaning of the variable you are actually analyzing." I think that this is generally desirable for many studies. I also agree that there is a tendency in the social sciences to overemphasize standardized coefficients and to not even report unstandardized coefficients. That is very unfortunate in my opinion, as I believe both are important and have their place.

That being said, there are fields (mine included: psychology) where we are dealing with variables that simply do not have an intuitive metric. Many variables are based on test or questionnaire sum or average scores. People use different tests/questionnaires with different metrics/scoring rules in different studies. What does it mean when, for example, subjective well-being is expected to change by 2.57 for every one-unit change in self-rated health and by 1.24 for every one-unit change in extraversion when self rated health is measured on a 0 - 10 scale and extraversion ranges between 20 and 50?

Standardized estimates can give us a better sense for the "strength" of influence/association in the presence of other predictors than unstandardized coefficients when variables have such arbitrary and not widely agreed upon metrics. The interpretation in standard deviation (SD) units is not completely useless in my opinion, especially since we operate a lot with SD units also in the context of other effect size measures such as Cohen's d. It allows us (often, not always) to see fairly quickly which variables are relatively more important as predictors of an outcome--we may not care so much about the absolute/concrete interpretation or magnitude of a standardized coefficient, but it does matter whether it is .1 or .6.

In addition, in the context of latent variable regression or path models (i.e., structural equation models), unstandardized paths between latent variables often have an even more arbitrary interpretation as there are different ways to identify/fix latent variable scales (e.g., by using a reference indicator or by standardizing the latent variable to a variance of 1). Regardless of the scaling of the latent variables, the standardized coefficients will generally be the same.

This does not mean that I recommend standardized coefficients over unstandardized coefficients. Variance dependence and (non-)comparability across studies/different populations are important issues/drawbacks of standardized coefficients. Unstandardized coefficients should always be reported as well, and they are very useful when variables have clear/intuitive/known metrics such as, for example, income in dollar, age, number of siblings (or pretty much any count), IQ scores, etc. Unstandardized coefficients are also preferable for making comparisons across groups/populations/studies that used the same variables. I would always report both unstandardized and standardized coefficients along with standard errors and, if possible, confidence intervals.

I believe there are many examples of regression or path models in psychology for which standardized coefficients were reported and that did advance our knowledge regarding which variables are more important than others in predicting an outcome.

Destroy Image Classification by Ensemble of Pre-trained models | by Vardan Agarwal | Towards Data Science

Dear @Sam Nicholson, I'm afraid climate models do not calibrate any parameter. These models are developed considering different physical processes in terms of their equations. The number of physical processes considered, the way in which they are analytically described and also numerically implemented, depends on different factors, including the spatio-temporal discretization of the climate model (i.e. grid dimension, time step length) and the total temporal horizon to be simulated. The evolution of the state variables of climate models depends on different model forcings such as the income radiative flux affecting the modeled system through the physical processes considered by the model (e.g. vapor condensation, evaporation, moisture recycling, etc. ). Therefore, to simulate the expected paleoclimate trends, which are often revealed by different proxies, you should include the corresponding climate forcings in the model, which are supposed to generate such paleoclimate trends.

With no doubt that blockchain has been great so far. It brought us cryptocurrency and was the technology that set the pace for a myriad of industry-disrupting innovations. If you know what blockchain is, you probably also know that it is a kind of distributed ledger.

The choice of a model should be based on underlying physical explanations first.

See the manual of my software for examples:

If you do any of the models that employ regression and particularly cointegration then the dynamic stability of your variables are more important than anything else ,this will be true for time series data. If you employ cross section data there are several econometric tests that can be performed to assure soundness of the mode and standard deviation is only one criterion that may be covered by the the various tests.

Thank you for your answer

Hi, check also Fixed-effect panel threshold model using Stata, Qunyong Wang, The Stata Journal (2015) 15, Number 1, pp. 121–134

That is a good question.

interesting. I did not know. Keep going!

Adnan Majeed, thank you for sharing. In the PDF, it's mentionned that fixed effect for a mean difference (likelihood = "normal" and link = "identity") should be working, but I noticed that "burn in" iteration are not taken into consideration (see my attached PDF). Other likelihood / link works well.

I also opened an issue on Github... I wonder if there is other way I could proceed for a fixed effect model in mean difference with that package (random effect works well) ?

You need to be mindful of what any one IC is doing for you. They can look at 3 different contexts:

(a) you select a model structure now, fit the model to the data you have now and keep using those now-fitted parameters from now on.

(b) you select a model structure now and keep that structure, but will refit the model to an expanded dataset (reducing parameter-estimation variation but not bias).

(c) you select a model structure now and keep that structure, but will continually refit the model as expanded datasets are available (eliminating parameter-estimation variation but not bias).

Dynamic panel models like Arellano bond (xtabond,xtdpdsys) are used for large N and small T. For small N and large T, you can use FE, or Zellner's seemingly unrelated regressions (SUR)-with Stata command sureg. or FGLS (xtgls).

I try to replicate your code, and I don't get "None", I just get an error when I try to do an inference with the model (see image-1). In your forward you do it:

def forward(self, xb):

xb = self.features(xb)

xb = self.avgpool(xb)

xb = torch.flatten(xb,1)

xb = self.classifier(xb)

return xb

but features, avgpool and classifier are "variables" of network, then you need to do:

def forward(self, xb):

xb = self.network.features(xb)

xb = self.network.avgpool(xb)

xb = torch.flatten(xb,1)

xb = self.network.classifier(xb)

return xb

then when I run the forward again, everything looks ok. (see Image-2)

If this not work for you, could you share your .py? I need to check the functions: to_device, evaluate, and check the ImageClassificationBase class. To replicate the error and be able to identify where it is.

Dear Gonçalo Maria ,

Please note that transforming variable(s) does NOT make the series stationary, but rather makes the distribution(s) symmetrical. Application of logarithmic transformation needs to be exercised with extreme caution regarding properties of the series, underlying theory and the implied logical/correct interpretation of the relationships between the dep variable and associated selected regressors.

Reverting to your question, the proposed solution would be to use the Autoregressive Distributed Lag (ARDL) model approach, which is suitable for datasets containing a mixture of variables with different orders of integration. Kindly read the manuscripts attached for your info.

All the best!

Daniel Marusig I agree with Anoop Kumar Srivastava

Psychological Councilling data of pateints can be analysed statistically - https://www.ncbi.nlm.nih.gov/books/NBK425420/

Thank you Remal Al-Gounmeein for sharing! I think it's interesting because I have somewhat the opposite problem that this paper addresses; many people in my field use simple correlation to relate the abundance of taxa to one another, but typically those covariances can be explained by an environmental gradient. So including covariates actually vastly decreases the number of "significant" relationships. But still it's a point well-taken because explaining that e.g. taxon1 and taxon2 don't likely interact directly even though they are positively or negatively correlated would in fact require presenting the results of both models. Thanks!

Thanks Adnan Majeed

Dear @Jesie, you are talking about a supervised model. It is known as the C4.5 algorithm developed by Quinlan. This kind of model implies trainig, pruning, and test stages. The trainig will perform a tree growing up. It will split data based on the best feature choice iteratively. It will conitnue until a) there are not more data, b) the minimum volume of data in a node is not enough for a node splitting, or c) all data into a node belong to the same target class

After training the model, a pruning stage is performed to avoid the overfitting in the model. A cross-validation technique (k-folds=10) should be used for better results. You could use the weka software for analysis of this algorithm. I hope it is useful for you. (https://www.cs.waikato.ac.nz/ml/weka/).

Econometric analysis of cross section and panel data by Jeffrey Wooldridge is a comprehensive reading on this topic (or even the most established textbook in this field).

Why don't you try a longitudinal models? the best intro is here:

Prof. Davidian developed these models. Feel free to ask questions.

Best, D. Booth

A new source of bioclimatic variables' uncertainty has been discovered that is related to the selection of the specific month/quarter (e.g. wettest quarter in case of bio8). Please refer to this paper:

Python library pandas will be useful here, just load the CSV and use the sample for analysis. This link would be helpful

Dear Murat,

I am more than pleased!

Ashay Singh

https://www.dataversity.net/database-reverse-engineering-case-study/

In general, to explore the heterogeneity in a panel data, you need to first identify the group classifications of data in line with variables of interest.

I suggest you to visit the website of TensorFlow , we can find you the example to explained you how devise your datasets and how you can test your model by using of images tests one by one.

I hope that be Claire for you.@ Nima Karimi

Dear Glückler,

Thanks for your answer. Yes, considering the calculation of the dating error for ²¹⁰Pb, the new rplum package give me a solution.

https://cran.r-project.org/web/packages/rplum/

Regards,

Mingda

I am an applied economist; not an econometrician. So I hesitate to say that published papers in good journals are not adhering to good practice. However, having made this gesture towards humility, this is what I am about to argue. The key to determining the model specification that best conforms to the underlying process or mechanism that generates your data (the data generating mechanism) is diagnostic testing. In the case of distinguishing between a static or a dynamic model - hence, making a judgement about the underlying data generating mechanism - a simple procedure is to estimate a static panel FE model and then test the within-group residuals for serial correlation. (In Stata, this may be done by the user-written xtserial package, which implements a test devised by Jeffrey Wooldridge.) If the null of no serial correlation in the residuals is not rejected, then it is reasonable to conclude that your model does not suffer from unmodelled dynamics. In this case, estimate a static model. However, if the null is rejected, then your model has not captured the dynamics in the data. In this case, estimate a dynamic model. In effect, estimating a dynamic model is to displace the dynamics in the data from the error term (where serial correlation violates the assumptions of the estimator) into the estimated part of the model (where the dynamics are explicitly modelled and thus yield useful information - e.g. enabling long-run and short-run effects to be distinguished). If this reasoning is correct, then it is not helpful (to say the least) to report both static and dynamic models. They cannot both be well specified. Indeed, it is my observation that researchers who report both do not undertake diagnostic tests. I suspect that if diagnostic testing were to be undertaken then it would reveal the static models to be misspecified and thus - by their construction - yielding biased and inconsistent estimates . (In my experience, time-series data usually exhibit serial dependence, which needs to be taken into account in specifying econometric models). To summarise: (I) test for serial correlation; and (ii) if you find it, explain to your readers why you specify and report only a dynamic model.

By the way, the GMM is a general approach to estimation. As such, it is often applied to dynamic models but is not restricted to dynamic modelling (e.g. the GMM approach can be used to estimate static models).

I agreed with A. U. Usman answer. But some other techniques like non-linear analysis, cluster analysis, factor analysis, etc. may be utilized in this regard

In recent updated versions such as ANSYS 19.3, Altair hyperworks, most of the cases, all input files formats are compatible with any solver and intermediately we can move from one solver to other without much hassle.

Richard John Smith,

I think you'll find these papers interesting:

when it comes to some kind of national, government maps, you may not find such a list

but RG is huge, maybe just people will tell you that their country is like in Canada or not

in Poland, unfortunately not, we have a geodetic map of Topographic Objects Database similar to OSM

You can run Fixed Effects, Random Effects, or Pooled OLS.

Variables such as population growth and foreign aid are inappropriate

If the dependent variable is stationary at I(1) you can use ARDL Bounds test, but if the dependent variable is stationary at I(0), you can use an augmented Autoregressive Distributed Lag (ARDL) bounds test.

The GMM dynamic panel estimators are appropriate for large N and small T and generally pre-tests are not conducted. A large panel sample size is often used so that the Central Limit Theorem can be invoked for the asymptotic normality of coefficients even if the residuals are non-normal. Robust standard errors can be employed to deal with autocorrelation and heteroscedasticity. For a broader discussion and how to apply GMM methods in Stata see:

Roodman, D. (2009). How to do xtabond2: An introduction to "Difference" and "System" GMM in Stata. The Stata Journal, 9(1), 86-136. With very small T unit root testing would not typically be applied. If T is large GMM estimators can become unreliable because the number of instruments becomes large and the instrumented variables can be overfitted and so may not remove the endogenous components of the lagged dependent variable(s) as intended. When N is not large and T is moderate you may wish to use a bias corrected LSDV estimator to deal with dynamic panel bias, although these assume that all variables other than the lagged dependent variable are strictly exogenous. To apply a bias-corrected LSDV estimator to a potentially unbalanced dynamic panel in Stata see:

Bruno, G. (2005). Estimation and inference in dynamic unbalanced panel-data models with a small number of individuals. The Sata Journal, 5(4), 473-500. With moderate T you may wish to apply panel unit root tests. Second generation tests that deal with cross-sectional dependence are recommended. An example paper that uses Bruno's estimator and panel unit root tests is given in:

(1) Goda, T., Stewart, C. and Torres García, A. (2019) ‘Absolute Income Inequality and Rising House Prices’, forthcoming in Socio-Economic Review. An example application of GMM dynamic panel estimators without unit root testing is: Matousek, R., and Nguyen, T. N., and Stewart, C. (2017) ‘Note on a non-structural model using the disequilibrium approach: Evidence from Vietnamese banks’, Research in International Business and Finance, 41, pp. 125 – 135.

Holger Steinmetz Thank you so much for your consistent help. I will go on reading and studying.

Hi.,

Google Trends to improve your SEO:

I've learnt from the responses so far. Thanks for the insightful contributions

Seyed Shayan Sajjadinia Many thanks for your help!

You can write me here ( message ) or to my email : Alireza.shabani@hotmial.com

Yes, this too is a dynamic model and applying Arellano-Bond (or the related GMM-type estimators) would be perfectly adequate.

Hi evan;

As far as i know, to control biases of tracking and surveys (whatever the method used to obtain presence data) datasets is to become them spatially and temporarily equal. That is, using standardizing your datasets to make these datasets comparable across scales.

On the other hand, given the different nature of our datasets, you can use the R package named "biomod2" which integrates different modelling approaches, namely GLM, Maxent etc for the same datasets separately or combined.

I strongly recommend you to check this out, indeed this modelling approach is helpful to control biases instead of using solely one method.

See this paper; https://www.sciencedirect.com/science/article/pii/S1470160X15002393

I hope I've been of help to you. However, I think that other researchers with huge backaground in ecologuical modelling could give you more useful advises.

Best regards

Jon

It differs according to which mixed GLM you are discussing

for binomial see

and the references it contains

for Poisson , see

In your panel data N<25 and T >25, So I would like to suggest you to use Pooled OLS, FE or RE (based on the huasmen test). And also you can apply cointegration regression models such as FMOLS and Dynamic OLS. If some independent variables are stationary at level, you should use the first differences of those. (You don't need to uplaod data here.)

The Durbin Watson in Panel data is not effective, to fixed effect you can see

and to random effects you can see:

Good luck

Hope this will help you. Please see the attached document.

Hazim Al Dilaimy What answers

@Noor

You can now try out the discussion forum hosted in Google spreadsheet as well as downloading it into your own Google drive to host your own discussion forums. For more details, please visit http://bit.ly/forumtoolbox

If you'd like to collaborate with me to conduct research using modified versions of the discussion forum, please email me at ajeong@fsu.edu to discuss your ideas!

Sincerely,

Allan

Mohammed Abdullah Al-Hagery if you go through this white paper, i think it will clear all of your questions regarding IPFS.

Thanks.

you should be able to fit your data after putting them in the correct form and calculating the required terms. you should use origin program for ease of calculation. please check the following articles:

Cross Linked Sulphonated Poly(ether ether ketone) for the Development of Polymer Electrolyte Membrane Fuel Cell

Read more articles in the area. 10 years is too much, reduce it. You can stop when the model is in best fit or passes diagnostic tests.

Dear Surojit,

You can scale your input data and the output with the method chosen by you or you can find the method the software makes it. That will allow you for inverted process i.e. scaling back the predictions to real values. It is important taking into consideration the prediction errors values. Having them calculated (MSE, MAPE or others) on the scaled values does not give much, but - after scaling back - we can say thet e.g. predicted price is ... $ with mean average percentage error of ... $. So the method of scaling should be known for scaling back purposes and than prediction errors can be shown in real values too.

Best regards

Hubert

P.S. one of my paper shows that the method of scaling can influence the predictions (end their errors) when ANN are used

Thanks for given a fruitful reference to identify the joint break points

Dear Saugat Bhattarai ,

I think you are using few samples to learning a ResNet network. In particular, this network requires a lot of samples during the training, due to its high capacity. I had similar problems in CIFAR-10 using ResNet 50, and when I used data argumentation, the network works fine. Finally, I recommend you to decrease the learning rate, iteratively, after some epochs, for instance, after 80, 120, 160, 180 epochs.

Thank you Moreno. I am actually using a dynamic panel now (arellano-bond system GMM). I think it addresses the data limitations problems.

An array of studies investigated OSM data and assessed the geometric, attributive and temporal accuracy, and completeness of the mapped features. Besides intrinsic approaches, most of these studies compare OSM data to established commercial or official geographic data on road networks [29–31], buildings [32], and land use data [33–35]. Their results show, first, that OSM data is only slightly inferior to official/commercial data in terms of accuracy. Second, OSM data completeness increases at a rapid rate and is assumed to have reached or exceeded the level of completeness of commercial data in the meantime. Third, the completeness of OSM is positively correlated to population density and can be considered to be particularly suitable for the spatial analysis of urban areas.

29. Haklay, M. How good is volunteered geographical information? A comparative study of OpenStreetMap and ordnance survey datasets. Environ. Plan. B Plan. Des. 2010, 37, 682–703. [CrossRef]

30. Girres, J.F.; Touya, G. Quality assessment of the French OpenStreetMap dataset. Trans. GIS 2010, 14, 435–459. [CrossRef]

31. Neis, P.; Zielstra, D.; Zipf, A. The street network evolution of crowdsourced maps: OpenStreetMap in Germany 2007–2011. Futur. Internet 2011, 4, 1–21. [CrossRef]

32. Hecht, R.; Kunze, C.; Hahmann, S. Measuring completeness of building footprints in OpenStreetMap over space and time. ISPRS Int. J. Geo-Inf. 2013, 2, 1066–1091. [CrossRef]

33. Arsanjani, J.J.; Mooney, P.; Zipf, A.; Schauss, A. Quality assessment of the contributed land use information from OpenStreetMap versus authoritative datasets. In OpenStreetMap in GIScience: Experiences, Research, and Applications; Arsanjani, J.J., Zipf, A., Mooney, P., Helbich, M., Eds.; Springer: Heidelberg, Germany; New York, NY, USA; Dordrecht, The Netherlands; London, UK, 2015; p. 324.

34. Arsanjani, J.J.; Vaz, E. An assessment of a collaborative mapping approach for exploring land use patterns for several European metropolises. Int. J. Appl. Earth Obs. Geoinf. 2015, 35, 329–337. [CrossRef]

35. Dorn, H.; Törnros, T.; Zipf, A. Geo-Information comparison with land use data in Southern Germany. Int. J. Geo-Inf. 2015, 4, 1657–1671. [CrossRef]

Hope that helps!

Hello Chingyeh, R2 alone cannot tell a good model. I recommend you look at the adjusted R2 which will correct or adjust the model with respect to the variables. Notice that if the variable does not correct significantly to the model, the adjusted R2 will reduce but the ordinary R2 will continue to increase as more predictors are incorporated into the model. Very importantly, look at the lack-of-Fit test result. If it is insignificant, then you have a good model. Still there is a last step where you have to test your model. If your model is for predictions, then look at the pred R2 which does not have to be too different from the adj R2. For a good range of adjusted R2, consider 0.8-0.99. Be careful with overfitted models.

Let's separate the technical issue from the economics one, which I think is what you want to know. Adding more data, such as going to quarterly (GDP data are not usually available in monthly frequency) data only addresses the technical issue of estimation without addressing the economic question. The average post-WWII business cycle in the U.S. is 70 months, or a little less than 6 years. So, 7 years of data will not be adequate to capture an average U.S. business cycle. It will most certainly not be able to capture any growth pattern. In order to have meaningful interpretation of your results, you will need to have a longer time span of data, and not just more data.

Yes the write answer is

fractional probit model can be used to analyze this kind of data.