Computational Data Mining - Science topic

You can select the best algorithm based on the measure of performance from a number of data mining algorithms. A exhaustive list may be found: https://www.kdnuggets.com/2015/05/top-10-data-mining-algorithms-explained.html

Other approaches exist. For instance (and mostly application based)

Extreme Data Mining.

A strategy to apply machine learning to small datasets in materials science.

Machine learning on small size samples: A synthetic knowledge synthesis.

These Papers will help you:

1: Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers: https://arxiv.org/pdf/1801.06889.pdf

2: Visual Analytics for Explainable Deep Learning:

3: CNN EXPLAINER: Learning Convolutional Neural Networks with Interactive Visualization :

Convolutional Neural Networks (CNNs) is the most popular neural network model being used for image classification problem. The big idea behind CNNs is that a local understanding of an image is good enough

Top 5 Classification Algorithms in Machine Learning

There are multiple articles on that special topic. the main challenge lies in the fact that there are multiple perspectives from which you can look at the problem: algorithmic (data science) including preprocessing, database, learning algorithms etc.; acceptance within an organization of AI-based systems including linked topics such as explainability (explainable AI- XAI), security of such AI-based systems (e.g. adversarial attacks). Depending on the persepctive you are interested in you might want to look up those terms I mentioned above. Furthermore there are some articles that try to summarize some of the challenges (I linked them at the bottom of this answer). Further more you can have a look at our recent papesr that deal with decision criteria for AI-based system in an industrial context as well as a summary paper on AI security and Explainable AI.

Decision Factors:

AI security:

xp

xEx

As far as the current scenario is considered noting can be an apt topic than, SARS-COVID 19. Try using your computational skills and expertise in this field.

Have a look to this paper

C. ROUSSEY,S. BERNARD, G. ANDRÉ, D. BOFFETY. Weather Data Publication on the LOD using SOSA/SSN Ontology.Semantic Web Journal, 2019 http://www.semantic-web-journal.net/content/weather-data-publication-lod-using-sosassn-ontology-0

Dear Colleagues and Friends from RG,

In my opinion, in the coming years, one of the key applications of artificial intelligence integrated with other Industry 4.0 technologies, including Big Data Analytics, will be improvement of information search on the Internet.

Conducted scientific research confirms the strong correlation between the development of Big Data technology, Data Science analytics, Data Analytics and the effectiveness of the use of knowledge resources. I believe that the development of Big Data technology and Data Science analytics, Data Analytics and other ICT information technologies, multi-criteria technology, advanced processing of large information sets, and Industry 4.0 technology increases the efficiency of using knowledge resources, including in the field of economics, finance and organization management. In recent years, ICT information technologies, Industry 4.0 etc. have been developing dynamically and are used in knowledge-based economies in particular. These technologies are used in scientific research and business applications in commercial enterprises and in financial and public institutions. Due to the growing importance of this issue in knowledge-based economies, an important issue is the analysis of the correlation between the development of Big Data technology and Data Science analytics, Data Analytics, Business Intelligence and the effectiveness of using knowledge resources to solve key problems of civilization development. The use of Big Data, Data Science, Data Analytics, Business Intelligence and other ICT information technologies as well as advanced data processing Industry 4.0 in the processing of knowledge resources should contribute to increasing the efficiency of knowledge resource processing in knowledge-based economies, including in the field of economics and finance.

In recent years, the scope of applications of Big Data technology and Data Science analytics, Data Analytics in economics, finance and management of organizations, including enterprises, financial and public institutions, has been increasing. Therefore, the importance of implementing analytical instruments for advanced processing of large data sets in enterprises, financial and public institutions, i.e. the construction of Big Data Analytics platforms to support organization management processes in various aspects of operations, including the improvement of customer relations, is also growing. In my opinion, scientific research confirms the strong correlation between the development of Big Data technology, Data Science analytics, Data Analytics and the effectiveness of the use of knowledge resources. I believe that the development of Big Data technology and Data Science analytics, Data Analytics and other ICT information technologies, multi-criteria technology, advanced processing of large information sets, and Industry 4.0 technology increases the efficiency of using knowledge resources, including in the field of economics, finance and organization management. In recent years, ICT information technologies, Industry 4.0 etc. have been developing dynamically and are used in knowledge-based economies in particular. These technologies are used in scientific research and business applications in commercial enterprises and in financial and public institutions. Due to the growing importance of this issue in knowledge-based economies, an important issue is the analysis of the correlation between the development of Big Data technology and Data Science, Data Analytics, Business Intelligence and the effectiveness of using knowledge resources to solve key problems of development of business entities. In recent years, the use of 5G technology to collect data from the Internet can significantly contribute to improving the analysis of sentiment of Internet users' opinions and the possibility of extending the use of research techniques carried out on Business Intelligence, Big Data Analytics, Data Science and other research techniques using ICT information technologies , internet and advanced data processing typical of the current fourth technological revolution referred to as Industry 4.0.

In recent years, organization management processes have been improved through the implementation of information technology and advanced data processing technologies Industry 4.0 into the IT analytical platforms Business Intelligence, Big Data Analytics, etc. The technologies of advanced analysis of big data sets Big Data Analytics and research processes carried out on Business Intelligence platforms are used also to improve business management processes. Data collection processes on the Internet can be supported by the use of 5G technology. In recent years, information technology management models in organizations have been enriched with advanced 4.0 industry data processing technologies, including cloud computing, Internet of Things, artificial intelligence, machine learning and more. The use of information systems in built models of information technology management in organizations, etc. is currently taking place in many areas of functioning of various types of business entities. The use of ICT information technologies and advanced data processing technologies i.e. typical for the current technological revolution Industry 4.0 already covers almost the entire functioning of business entities, from computerized sales support systems, logistics, accounting, reporting, risk management to marketing activities on the Internet and designing new products and innovative solutions in information systems. Online banking is starting to dominate, whose development is determined by technological progress in the field of ICT and Industry 4.0 information technologies. Computerization is also increasingly affecting public sector institutions servicing tax systems and settlements of business entities. Business Intelligence analytical platforms have also been developed for several years in the SME sector. Business Intelligence systems supporting analytical processes and organization management are produced by IT companies not only for large corporations. The analyst of large information sets in Big Data databases is also developing. Big Data Analytics and Data Science analytical systems are used by more and more types of business entities to analyze both the markets in which they operate and complex processes that are conducted or diagnosed and researched in these enterprises. Computerization also covers financial and economic risk management processes, etc. In all these areas of ICT technology application, building and improving IT technology management models in organizations is also an important issue. Therefore, specific information technology management models should be tailored to the specifics of the operations of a particular business entity, enterprise, company, corporation, public institution or financial institution.

On the other hand, the collection of large data sets about users of specific websites and portals in Big Data database systems generates new categories of information security risk. The database of social media portal such as Facebbok is already a powerful collection of information. Some research centers specializing in the use of large data sets Big Data downloaded from social media portals through the analysis of sentiment I prepare reports that can be helpful in forecasting phenomena and processes in the future. Medicine is one of the areas where there are great opportunities in this matter. For example, insurance companies and commercial banks that grant loans may be interested in information posted by users on Facebook and possibly also on other social media sites. Apparently, some insurance companies and commercial banks during the analysis of an application for insurance or credit are looking at the information content of accounts, applicant profiles, potential client, contractor posted on social media portals.

Another area of ??application of analytics carried out on large data sets collected in Big Data database systems is sentiment analysis in the field of surveying the opinions of Internet users regarding specific products and / or services and companies producing them. Large amounts of information downloaded from comments, entries, posts from social media portals are processed in Big Data database systems to determine, e.g. consumer awareness regarding the offer of products and services of specific companies. This type of information is of great importance for the purpose of planning advertising campaigns informing about the mission, idea, product offer, usability features of a given company's offer. This type of data may be relevant to forecasting changing consumer preferences for specific companies' offers. Techniques for collecting analytical data on the Internet can be supported by the use of 5G technology.

I am also involved in research on knowledge management using the Big Data computerized database platforms. In my publications available on the Research Gate portal, I described the key determinants of the development of Big Data technology and the security of information obtained from the Internet, collected and processed in Big Data databases. I also described the development of analytics using Business Intelligence platforms that are used in enterprises. Business Intelligence based analytics, as well as Data Science and Big Data Analytics are increasingly being used to improve business management processes. The development of this analytics based on the implementation of ICT and Industry 4.0 information technologies into analytical processes has a great future ahead of it in the coming years. I invite you to cooperation.

One of the areas in which the possibilities of market analytical technology applications, including data downloaded from internet portals, are growing is the marketing of enterprises and institutions. In recent years, the development of marketing is determined by the development of Industry 4.0 technology and the development of open innovations on the Internet. Open innovations developed on the Internet concern, among others, free information and marketing services. The issue of the possibility of publishing specific content, texts, banners, comments etc. on the Internet and obtaining free information are key determinants of the development of information services on the Internet. On the other hand, the largest internet technology corporations earn income mainly from paid marketing services. Therefore, the Internet environment is a kind of mix of free and paid information and marketing services, which are simultaneously, simultaneously and simultaneously interrelatedly developed by various Internet companies. Currently, research is conducted into the analysis of the development of open innovations in the field of free information services, which are the main factor of business success of the largest online technology companies, which include such concerns as Google and social media portals such as Facebook, Instagram, YouTube, Tweeter, LinkedIn and others .

The development of internet information services will be determined by technological progress in the field of new ICT, communication technologies and advanced data processing techniques typical of the current technological revolution referred to as Industry 4.0. The development of information processing technology in the era of the current technological revolution called Industry 4.0 is determined by the use of new information techniques, for example in the field of e-commerce and e-marketing. These solutions are the basis for the business success of the largest online technology concerns that offer information search, data collection and processing services in the cloud (e.g. Google) and provide information services on platforms developed in social media portals (e.g. Facebook, Instagram, YouTube, Tweeter, LinkedIn, Pinterest, and more).

The current technological revolution referred to as Industry 4.0 is motivated by the development of the following factors: Big Data database technologies, cloud computing, machine learning, Internet of Things, artificial intelligence, Business Intelligence and other advanced technologies of Data Mining.

The information technologies mentioned above, combined with the improvement of ICT and communication technologies, along with the progressive process of increasing the computing power of computers will become an important determinant of technological progress in various branches of industry in the coming years. Based on the development of these new technological solutions, the processes of innovatively organized analyzes of large information collections gathered in Big Data database systems and computing cloud computing for the purposes of applications in such fields as machine learning, Internet of Things, artificial intelligence have been dynamically developing in recent years. , Business Intelligence. To this can be added other areas of advanced technologies for analyzing large data sets, such as Medical Intelligence, Life Science, Green Energy, etc. Processing and multi-criteria analysis of large data sets in Big Data database systems is performed according to the V4 concept, i.e. Volume (meaning large number of data), Value (large values ??of specific parameters of the information analyzed), Velocity (high speed of new information appearing) and Variety (high information diversity). The above-mentioned advanced technologies for processing and analyzing information are increasingly used for the needs of marketing activities of various business entities that advertise their offer on the Internet or analyze the needs in this regard reported by other entities, including companies, corporations, financial and public institutions. More and more commercially operating business entities and financial institutions conduct marketing activities on the Internet, including on social media portals. The possibilities of collecting market data on the Internet in subsequent years can be significantly expanded by using 5G technology.

The information and communication technologies listed above, combined with the improvement of ICT technologies and the implementation of Business Intelligence analytics into the processes of economic and financial, economic, macroeconomic and market analyzes may be instrumental instruments helpful in the efficient and effective management of economic, investment processes and enterprises, including analyzes carried out for the purposes of improving marketing activities in enterprises. More and more companies, banks and other entities need to carry out multi-criteria analyzes on large data sets downloaded from the Internet describing the markets in which they operate and contractors and clients with whom they cooperate. On the other hand, there are already specialized technology companies that offer this type of analytical services, prepare commissioned reports, which are the result of such multi-criteria analyzes of large data sets obtained from various websites and from entries and comments contained on social media portals. An important research technique that has been developing in recent years, the effects of which are used for the purposes of marketing activities of companies, is sentiment analysis carried out on large data sets collected from the Internet and stored in Big Data database systems.

In order to group the behavior of social media users into specific classes of behavior, these classes must first be defined. Sentiment analysis using large data sets collected from entries and comments from social media portals and transferred to Big Data database platforms can be helpful. Then, when observing the changes in certain types of behavior of users of social media portals, you can analyze the data collected in Big Data according to these observations. In addition, a useful tool can be an analysis of the behavior of users of social media portals based on current posts, entries and comments on specific social media pages, statistical analysis of comments on specific topics of posts. This type of research is carried out by online technology companies that run social media portals and use the results of these studies to develop their viral marketing services, because this field of marketing is a key determinant of revenue generated by these companies from advertising sales on social media portals. The basis of marketing activities conducted in this way are market research conducted by collecting market data from the Internet regarding the offer of individual companies, their competition, demand for specific products and services from Internet users as well as collecting, processing and analyzing this data in Big Data Analytics database and analytical systems. The process of collecting market data from specific websites can be improved by using 5G technology.

Industry 4.0 technologies are also used in the development of transaction systems and transaction security in the field of e-commerce and online banking. The key determinants of the globally developing e-commerce relate primarily to the implementation of ICT information technologies and advanced data processing technologies, i.e. industry 4.0 typical for the current technological revolution to computerized, automated transaction systems supporting online trading. In addition, the use of blockchain technology for transaction security systems and data transfer on the Internet. The use of ICT information technologies and advanced data processing technologies i.e. typical for the current technological revolution Industry 4.0 to online transaction systems supporting e-commerce already applies to almost all the functioning of online stores, from computerized sales support systems, logistics, accounting, reporting, risk management to Internet marketing activities and improving security systems for online transactions. Another important determinant of e-commerce development is the development of online mobile banking available on mobile devices and new solutions related to the Internet of Things technology. Online banking is starting to dominate, whose development is determined by technological progress in the field of ICT and Industry 4.0 information technologies. Computerization is also increasingly affecting public sector institutions servicing tax systems and settlements of business entities. In addition, Business Intelligence analytical platforms supporting the management processes of companies operating also in the e-commerce sector have been developed for several years. The analyst of large information sets in Big Data databases is also developing. Big Data Analytics and Data Science analytical systems are also used by businesses operating also in the field of e-commerce.

In recent years, new internet marketing instruments have also been developed, mainly used on social media portals, and are also used by companies operating in the e-commerce sector. Internet technology and fintech companies are also emerging that offer information services on the Internet to support marketing management, including the planning of advertising campaigns for products sold via the Internet. To this end, sentiment analyzes are used to survey Internet users' opinions regarding dominant awareness, recognition, brand image, mission and the offer of specific companies. Sentiment analysis is carried out on large data sets downloaded from various websites, including millions of social media sites collected in Big Data systems. The analytical data collected in this way are very helpful in the process of planning advertising campaigns carried out in new media, including social media portals. These campaigns advertise products and services sold via the Internet, available at online stores. In view of the above, the development of e-commerce is determined mainly by technological progress in the field of ICT information technologies and advanced data processing technologies Industry 4.0 and new technologies used in securing financial transactions carried out via the Internet, including e-commerce related transactions, e.g. technology blockchain. I have described the above issues of various aspects of the application of information systems and ICT, including Big Data, Business Intelligence in companies operating on the Internet in my scientific publications available on the Research Gate portal. I invite you to cooperation.

According to the above, in my opinion, the use of 5G technology to collect data from the Internet will significantly contribute to improving the analysis of sentiment of Internet users' opinions and the possibility of extending the use of research techniques carried out on Business Intelligence, Big Data Analytics, Data Science and other research techniques using information technologies ICT, internet and advanced data processing typical of the current fourth technological revolution referred to as Industry 4.0. At present, however, all the potential applications of 5G technology in economic and other applications are unknown. These applications will be wide in both business processes carried out by technological internet companies as well as by security institutions. Globally operating technology internet companies, thanks to the use of 5G technology in research processes, will improve their offer of information, internet and marketing services addressed to Internet users. On the other hand, national security institutions and IT systems risk management departments operating in companies can also obtain a tool enabling a significant improvement of instruments ensuring a high level of security of information transferred via the Internet and other cybersecurity issues. Therefore, research on cyber security and e-commerce will be expanded to include the impact of 5G technology on the development of many aspects of these areas of activity of business entities, institutions and citizens increasingly using the Internet in various areas of business.

In view of the above, in my opinion in the coming years one of the key applications of artificial intelligence integrated with other Industry 4.0 technologies, including Big Data Analytics, will be improvement of information search on the Internet.

Best wishes.

Dariusz Prokopowicz

I work in the same topic and I use

https://www.mathworks.com/matlabcentral/fileexchange/52867-apriori-algorithm-for-association-rule-mining

it works with small dataset but with large database like mushroom or BMS1 or other datasets it takes time.

who has solution about that?@ Shafagat Mahmudova

DBMS / Relational DBMS .

I have read the question carefully and what you have wrote in brackets does not mean singleton species.

Refer this link... https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/746353/Algorithms_econ_report.pdf

If you are interested in the general properties of your data set, like any multivariate data set with non-trivial signals encoded in a matrix, its strength to infer groups, you should give network-based exploratory data analysis a try.

You'll find a bunch of related posts with ideas and links to literature on the Genealogical World of Phylogenetic Networks (GWoN)

Especially the neighbour-net is a most versatile tool for multivariate analysis, however, largely unknown and massively underused. Its applications in phylogenetics or linguistics are multi-fold but it can be very revealing for entirely different data (you didn't mention whether you are working with biological or other data), a few of the examples we explored on GWoN:

A network of breat sizes (to test a claim) – https://phylonetworks.blogspot.com/2018/01/false-reports-of-us-womens-breast-sizes.html

A network of gun legislation in the U.S. (to illustrate diversity) – https://phylonetworks.blogspot.com/2018/03/visualizing-us-gun-laws.html

A network of moons (for the purpose of classification) – https://phylonetworks.blogspot.com/2018/06/to-boldy-go-where-no-one-has-gone.html

A network of party programmes (to illustrate overall similarity and dissimilarity, also over time) – https://phylonetworks.blogspot.com/2018/10/jumping-political-parties-in-germanys.html

A network of playmates' change over time – https://phylonetworks.blogspot.com/2016/10/changes-in-playboys-women-through-60.html

Visualisation of the data behind rankings, e.g.

bikability and public transport in U.S. cities – https://phylonetworks.blogspot.com/2019/03/which-us-cities-are-best-for-walking.html

world happiness – https://phylonetworks.blogspot.com/2018/07/a-network-of-world-happiness.html

Hello please check MATLAB and Python software's.

Yes , for sure, this application has already made

Dear Ibrahim Musa I'm interested in using the PDMC dataset for my thesis. Could I get a copy too? Thank you!!

hello,

please use any statistical forecasting techniques and check which one is suitable for your data.

good luck@

Hi Umair Arshad,

Please follow the papers below.

1. Srivastava, S. K., Gupta, R., & Singh, S. K. (2018). Simple Term Filtering for Location-Based Tweets Classification. In Speech and Language Processing for Human-Machine Communications (pp. 145-152). Springer, Singapore.

2. Liu, R., Cong, G., Zheng, B., Zheng, K., & Su, H. (2018, July). Location Prediction in Social Networks. In Asia-Pacific Web (APWeb) and Web-Age Information Management (WAIM) Joint International Conference on Web and Big Data (pp. 151-165). Springer, Cham.

3. Ozdikis, O., Ramampiaro, H., & Nørvåg, K. (2018, March). Spatial Statistics of Term Co-occurrences for Location Prediction of Tweets. In European Conference on Information Retrieval (pp. 494-506). Springer, Cham.

4. Stowe, K., Anderson, J., Palmer, M., Palen, L., & Anderson, K. (2018). Improving Classification of Twitter Behavior During Hurricane Events. In Proceedings of the Sixth International Workshop on Natural Language Processing for Social Media(pp. 67-75).

5. Li, P., Lu, H., Kanhabua, N., Zhao, S., & Pan, G. (2018). Location Inference for Non-geotagged Tweets in User Timelines. IEEE Transactions on Knowledge and Data Engineering.

Thanks,

Sobhan

Dear Asif,

Please have a look at these python packages,

and

Documentation of Biopython can be found at

Hope this will help you.

It is same because correctly classified instances means the sum of TP and TN. Similarly, incorrectly classified instances means the sum of FP and FN. The total number of correctly instances divided by total number of instances gives the accuracy. In weka, % of correctly classified instances give the accuracy of the model. Cheers!

weka , rapid miner, sklearn python library are the easy to use for classification

Perhaps, you should concentrate on filtering models. Also, consider a couple of parallel corpora of a different nature.

You can use aforementioned datasets or if you want to scrap the data yourself there is Facebook graph API.

PYLON provides access to previously unavailable Facebook topic data and has some price.

If you don't exclusively want Facebook datasets you can easily get data from other sources like (twitter, google, Wikipedia) using pattern.

Basically it is a topic classification problem and it has many nuances.  Random Forest and SVM have been giving quite good results. However please note the pre-processing steps, feature selection technique and text representation scheme ( Bag of words, Topic Model, n-gram) will have a bearing apart from the model you are choosing.

The sole purpose of clustering is to detect clustering or grouping structure. K-mean algorithm can be converted easily to incremental clustering more details can be found here https://www.cs.princeton.edu/courses/archive/fall08/cos436/Duda/C/sk_means.htm

I think it is a good idea to apply batch clustering first. Then compare it with incremental clustering.

You may follow this papers for incremental clustering algorithms....

You can also contact the TPC at www.tpc.org They are coming up with an IoT benchmark which includes a kit that generates artificial data. The kit is still under development but you can show your interest.

Hello Salih,

I think Gene Expression microarray is a good choice for you, the features may range from 6000 to 60000 feature 

Disclaimer: I have worked for a company working on both Amazon Web Services and Azure machine learning toolboxes. At the time of writing: Azure machine learning studio is richer wrt. capabilities, a comparison which was found during comparing the tools in aforementioned company. Amazon might catch up, but currently I am not seeing that happen. Happy to hear if someone disagrees with me, I myself work on AWS all of my working days..

Ease of use: please go and test it and compare it to tools you mentioned. Frankly, I dont get it how GitHub is included in a list of machine learning tools...GitHub and BitBucket are both online code version control repository offering services..not ML tools...

if you need Matlab please contract me

You should try looking at the data sets in the Machine Learning Repository as there are many different ones there. Plus the bonus is that other people will be able to use the data set to repeat your experiments if you publish. Also you will be referred to previous users of the data set you choose. http://archive.ics.uci.edu/ml/ 

Your data set is imbalanced as your class is not a 50/50 or 60/40 distribution. If you use decision trees you might not need to balance your data set. Otherwise you can use under sampling (i.e. use all the smaller class and randomly select same number of majority class several times to make multiple data sets and then combine all classification results) to balance it this is best way for this without loosing information. If you use boosting you could alter the weights and balance data that way. As mentioned above the best way to decide is to explore the data fully. If any other attributes are also imbalanced over their values this will also affect classification results.

Ok, thanks everyone..I am gong through the 2 docs from @Dmitri and @Mateo. I will start with KNN model and evolve this further. Will keep you posted.

Hi Johnson,

Several ways to do that; for example, you can use WEKA's KnowledgeFlow as follows:

Training set will be passed into SimpleKMeans, after that prediction model gets built using J48 based on the generated clusters. All this process is evaluated using cross-validation.

NB: The attached image shows the flow process where FilteredClassifier is containing both cluster and classification algorithms.

HTH.

Samer

maybe this can answer your question

Dear Hamdi I appreciate your help, this interval just simple example to show how to convert certain numerical attribute  value into uncertain numerical attribute as u know each attribute has many values and the rang between upper and lower bounds are not same for all values, so maybe the suggestion that split the intervals will work but the problem may will face within the classifer how they deal with two attributes as one.

Hi

you can used LIBSVM which is developed in c program.

Oh. I get it now. Thank you very much! :D

Lets assume we have 3 classes and 80 objects. 19 objects are in class 1, 21 objects in class 2, and 40 objects in class 3 (denoted as (19,21,40) ).

The Gini index would be: 1- [ (19/80)^2 + (21/80)^2 + (40/80)^2] = 0.6247 i.e. cost_before = Gini(19,21,40) = 0.6247

In order to decide where to split, we test all possible splits. For example splitting at 2.0623, which results in a split (16,9,0) and (3,12,40):

After testing x₁ < 2.0623:

cost_L =Gini(16,9,0) = 0.4608

cost_R =Gini(3,12,40) = 0.4205

Then we weight branch impurity by empirical branch probabilities:

cost_x1<2.0623 = 25/80 cost_L + 55/80 cost_R = 0.4331

We do that for every possible split, for example x₁ < 1:

cost_x1<1 = Fraction_L Gini(8,4,0) + Fraction_R Gini(11,17,40) = 12/80 * 0.4444 + 68/80 * 0.5653 = 0.5417

After that, we chose the split with the lowest cost. This is the split x₁ < 2.0623 with a cost of 0.4331.

You may follow these following articles for item similarity in data mining.....

file:///C:/Users/sanjay%20chakraborty/Downloads/www10_sarwar.pdf

file:///C:/Users/sanjay%20chakraborty/Downloads/V2I6-IJERTV2IS60377.pdf

thank u Mr.Alaa Khalaf

there are several health related datasets available publicly, you can look at them and try applying machine learning/modeling. they will serve as an excellent starting point

I know the most realistic division can be achieved by taking 10% of data for testing and 90% for learning to avoid such problems as overfitting and learning failure. You can consult my dissertation on opinion mining and lexical affect sensing for list of corpora I took for this purpose.

Dear Girish Babu M CGirish Babu M C,

I had done some similar tasks with R programming. In that case we extracted from data base, the factors of the points where we knew that (for example) rice/sugarcane exist . Then by knowing those factors we made a linear regression model by "gml" function. At the end we implemented the model to the image and it illustrated the locations where the existence of rice/sugarcane where predicted

Be success

Ali Madad

Thanks a lot Sir.

1- The New York Times Annotated Corpus

2- Clueweb dataset

3- TREC Web track dataset

loop testing algorithm are suitable

Hi Jose,

This UCI dataset may be useful for your experiements: https://archive.ics.uci.edu/ml/datasets/SPECT+Heart

Kind regards,

Olivier PARISOT

Thank you Laurent for the informations. I will give an eye on it. Hope it will be user friendly because my knowledge in programing is limited. Regards

Dear

please i want dataset for large graph for graph mining in matlab 

Thanks a lot 

thanks really helped me

Dear Yaakov and Amina,

thanks for your answers. Your papers are very interesting and gave me a good overview concerning the several techniques.

First i calculated the tfidf for the whole collection in order to get the most specific words of the corpus.

Now i calculate the tfidfs for every document and choose the top n elements depending on the tfidf value. Now i have to evaluate a "good" value for n.

I hope this approach leads to a good result.

Here is one example

You have to look at stacked generalization: http://www.cs.utsa.edu/~bylander/cs6243/wolpert92stacked.pdf

Hello,

Data-mining is part of the Knowledge Discovery Process presented by Fayyad. So I would advise to study first the whole framework in order to get a better understanding of what data-mining is. 

To my understanding, data-mining is just about using some kind of algorithm (closely associated with machine learning) in order to retrieve patterns from your data.

Data-mining is bound to your data. By knowing the type of data you want to mine, you may get insight of the kind of data-mining you need. For example, if you use textual data as input, you will have to focus on "text-mining". 

Hope it helps a bit.

Refer to this study:

and follow these tutorial guides

Hope they help.

Dear Catalin, 

One easy method would be to take a sample from the new generation in a befitting way for surveying the degree of implication into city life by individual's social media usage.

Hope this could be useful.

Regards- 

yes it is used in bat algorithm

Probably you have to check the memory status during execution of command as.matrix(), given your operational system (for windows leave open the task manager). What is your RAM in GB? Are you using 32 or 64 bit computer?

This depends on your capability in terms of how you are able to manipulate things. To make this clear to you, Mahout is a machine learning tool that allows you to execute a number of algorithms, however, mahout was developed to work on top of hadoop, which means as time goes on you may need to write your little codes or manipulate existing codes to address some problems. This means that you will face the challenge of writing mapreduce codes to solve your problem. However, spark has addressed this problem of having to face the headache of writing mapreduce codes always. Spark support lots of machine learning algorithms that you can use and they are easy to implement. Therefore, I will advise you go for it. You can check spark in action by visiting this link:

Like I said, it all depends on the problem you are trying to solve and your capability in handling the tools. You can also use ECL-ML (https://hpccsystems.com/download/free-modules/ecl-ml) to process large volume of data. In addition, do not also forget that WEKA, the popular machine learning tool support distributed data mining. This means that you can use a lot of algorithms on WEKA in distributed mode (http://weka.sourceforge.net/packageMetaData/distributedWekaHadoop/index.html).

All the best.

Or alternatively, you can also use RapidMiner. WEKA is good, but RapidMiner offers you better GUI.

Cheers,

Diky

It really depends whether you know the ground truth clusters or not.

In the first case, you could proceed with "external clustering quality metrics". A widely used metric in this case is Purity. But you can proceed with others, like Homogeneity, Completeness, V-Measure, Precision, Recall, Normalized Mutual Information and CMM (for streaming environments).

In the second case, it is, no ground-truth is known, you could proceed with metrics that account for the distribution of data, and nothing besides it. These metrics are called "internal clustering quality metrics". Examples include: Silhouette coefficient and Sum of Squared Distances (SSQ).

My master's dissertation covers these subjects at least partially, but it is in portuguese. I will leave a link here in case someone else needs it too.

Dear Mayur, here is the link to the clustering validation in general:

I have recently published a paper on clustering in banking, by using SOM methodology, and we used a predetermined methodology provided by the Viscovery (software that we have used). However, we have provided a logical validation of the clusters. It would be even better if you can apply a validation by the experts from the field, and also present this in the paper (we do not have this in our paper). 

Here are the links:

Bach, M. P., Juković, S., Dumičić, K., & Šarlija, N. (2014). Business Client Segmentation in Banking Using Self-Organizing Maps. South East European Journal of Economics and Business, 8(2), 32-41.

Do you want to derive a rating for items over all users' clicks or evaluate all items for each user separately?

if doing evaluation for each user separately, the best way is nothing other than dividing by max clicks in my opinion.

Have you also checked http://translatorsavenue.com/article_details.php?id=248&language_id=1

It is Arabic-English, English-Arabic and German-Arabic??.

Usually, in situations like this you need to install "special" package for your Linux distribution you are using that contains most of the "headers" required. One can also help is an installation of "dev" variant of solution you are trying to install.

It is possible to reformulate physics formulas such that the parameter spaces that are used by the functions, which represent fields are Euclidean spaces instead of spacetime geometric data spaces that have a Minkowski signature. When formulated with this much simpler parameter space most equations of GRT will become simpler and much more compact. For example with this choice the multi-dimensional differentiation operator can then be treated as a multiplying operator. 

I think the challenge is on getting to some reasonable metric. By way of example, take the interpretation objectives which Benno suggested (which I also  think is a good feature to include). Interpretation objectives has to be defined in a way in which it can be separable by the classifier given a training dataset for a supervised learning algorithm, a task which I think has nothing to do with ML but with human nature and the task at hand (not impossible but difficult).

Just a thought

to setup a var model , one need to make sure that:

1. the variables are stationary by differentiating. This is very important as any linear regression might result into spurious association due to non stationary.  

2. if there is seasonal effect  its possible to either add seasonality in your model or filter out seasonality using census x13. Differentiating to remove seasonal effect  or adding seasonal effect might influence your result in case if some one trying to infer causal relation.   

3. selecting best lag order. For me it is the most difficult part.  

 i) some one can either use information criterion like AIC, HQ to select lag order

ii) start with a lag order based on theory or knowledge of the field and keep increasing the lag order. 

4. After selecting the lag order , test for auto-correlation in the residuals using Portmanteau test or  Lagrange multiplier test. If there exist autocorrelation change the lag order by one until there is no autocorrelation. 

5. You can also check for heteroskeadsticity and normality test for the residuals. 

6. Prior to all of this transform data to log to remove non-linearity if exist like exponential?

there steps i would follow but im not sure .. so its your own risk if you take my advice:p .. you can test for cointegration, if exists its possible to setup vec model with out differentiating the data in level where its non stationary.  one of the good thing about vecm is it includes long term association besides the short term ones. :p but again i m just doing masters  :p I am still learning 

Why don't try purity, entropy, accuracy and mutual information?

For more information about these mesures, you can consult the following thesis.

Good luck.

Linear min-max normalization: (max-X)/(max-min). Also check if (min==max)

Have a look at Cross Industry Standard Process for Data Mining (CRISP-DM):

It is more concerned the data mining process & project.

thanks for answer

doesn't work in my case. See PM