Statistical Pattern Recognition - Science topic

Perhaps it can help

Dear Pedro, and remaining question followers, please correct me if I’m wrong but, as far as I understand Procrustes analysis as well as convolution 2D approaches do not take into consideration the kinetic nature (or order) of the points defining a shape (or trajectory). Take, for instance, the following two scenarios consisting of 4 points each (all the 4 points have the exact same coordinates but in a different order, see attached figure):

Scenario 1: A(x=0, y=0), B(x=0, y=1) , C(x=1, y=1) , D(x=1, y=0)

Scenario 2: A(x=0, y=0), B(x=1, y=1) , C(x=1, y=0) , D(x=0, y=1)

The fact that the shape defined by the 4 points is exactly the same (actually a square), the trajectory (defined as the time dependent location/positioning in the 2D plane) or the placement of objects (as in the question of Antonio) are quite dissimilar, as there is no geometric transformation able to convert one scenario into the other. Please share your thoughts/feedback on the topic.

Best regards, Luis

See our published paper on the topic: Cross-sex genetic correlation does not extend to sexual size dimorphism in spiders

As an associative memory-like neural network extensively used to solve robot motion control problems, let us cite the biologically-inspired Cerebellar Model Articulation Controller (CMAC) invented by Albus in 1975. For further details, follow:

It is easy to prove that (quote wikipedia):

Thank you Veli-Matti, I am going to try that code now, I am at the moment trying to use the adehabitat package, trying to convert the dataframe into a ltraj list to calculate distances, times, and angles and see if I can make it work as well with your code.

Thank you very much.

Daniel G.

A sensing matrix maps input vector to measurement vector through linear wighted summation of input. What makes a specefic matrix good, is application dependent. Now, both distributions more or less satisfy RIP. However hardware implementation of the Bernoulli matrix (binary or bipolar) is much much easier especially in analog domain. A Bernoulli wight is either 0 or 1 (or -1/1 in case of polar Bernoulli), but a Gaussian wight is a floating point figure. Multiplication of a flouting point number either in digital or analog, is resource consuming, while multiplication of a Bernoulli wight is feasible through implementation of a simple switch in analog domain or and instruction in digital. As an example consider RMPI analog to information devices which compressively sample the signal in analog and then reconstruct the signal in digital. Prior to quantization the sensing matrix should be applied, were through incorporating a Bernoulli matrix, the mulpliers are implemented as simples switches

Dear Mandal,

Thanks for your answer. I am using class specific dictionaries. Say for example three dictionaries (D1,D2,D3) where they represent three classes and I have a test signal of class (D2). How I would assign the test signal  based on class voting assignment ? 

.

i tend to formalize ML classification through the canonical analysis framework

(i admit this is a somewhat larger perspective but i believe it allows to have a simple picture)

in canonical analysis, you have individuals (say areas of land) described by two sets of features (say, the animals living in the area on the one hand, and the plants growing in the area on the other hand)

the aim of canonical analysis is to find linear factors in each description space so that factors are maximally correlated across description spaces ; for the land area example, this means finding (mutually orthogonal) combinations of animal population (linear factor in animal description space) strongly correlated to (mutually orthogonal) combinations of plant population (linear factor in animal description space) 

if you imagine the correlation matrix of the full description of the areas, it has a block structure

A ; B

^tB ; C

with A the correlation matrix of the animal description, C the correlation matrix of the plant description and B the cross correlation matrix of the animal-plant descriptors

while PCA uses all this correlation matrix, canonical analysis concentrates only on the B submatrix (indeed, canonical analysis technically boils down to a svd of B)

.

all this handwaving introduction to indicate that there are two kinds of correlations when you have two description spaces (which is the case in multi-label problems : you have a feature space and a label space) : correlations within each description space (which are not the objective of the analysis : you are not interested in correlations between labels stricly speaking if such correlations have nothing to do with your feature space) and correlations across description spaces (which are the objective of your analysis)

for instance, some labels might well be just correlated noise from the perspective of your description space : PCA would take such correlation into account but without any gain from the classification perspective ; canonical analysis will ignore such correlations and concentrate on cross-correlations between your feature space and your label space

.

Once you have performed a pair-wise comparisons and got the inter-object distance matrix, it is often useful to visualize results in form of a 2D scatterplot using multi-dimensional scaling (MDS) method. This may inspire you for more ideas beyond the K-means.

Hi,

I think this link may be useful for you

http://www5.cs.fau.de/activitynet/benchmark-datasets/daliac-daily-life-activities/ 

function df=dscr_fourier(matrice_contour)

% Calcul des descripteurs de Fourier correspondant a un contour donne

%

% df=dscr_fourier(matrice_contour);

%

% matrice_contour - matrice Nx2, ayant sur la premiere colonne les abcisses des

% points qui definissent le contour et sur la deuxieme

% colonnes leurs ordonnees

% df - vecteur des descripteurs de Fourier

%

map=matrice_contour;

tgt=map(:,1)+1i*map(:,2);

medie1=mean(tgt);

tgt0=tgt-medie1;

% recuperer les angles en radian pour faire le tri.

phpb=angle(tgt0);

phpb(phpb<0)=phpb(phpb<0)+2*pi;

[phpbs,idx]=sort(phpb);% trier la liste pour recuperer les indice afin de le faire pour le nombre complexe

%changer l'ordre ::inverser l'ordre

idx=flipud(idx);

% reorganiser les nombre complexe de haut vers le bas cd ordre descroissant

tgt=tgt(idx);

tgt0=tgt0(idx);

tgt0=tgt0+medie1;

vpb=tgt0;

m=length(vpb);

vpb=[vpb;vpb(1)];

l=[0,cumsum(abs(diff(vpb))).'];

b=[diff(vpb)./abs(diff(vpb))].';

L=l(m+1);

for n=1:m

cf=(L*(2*pi*n/L)^2)^(-1);

df(n)=0;

for k=2:m+1

df(n)=df(n)+b(k-1)*(exp(-1i*n*2*pi*l(k)/L)-exp(-1i*n*2*pi*l(k-1)/L));

end

df(n)=cf*df(n);

end

df=df.';

df=df/df(1); % normalisation pour similitude

df=df(2:m); % pour rendre invariant par changement d'echelle

df=abs(df);

Thanks all for your comments.

I am going to use Dr. Vikas option.

I have a try,it's ok.maybe you need to have a check about the data type

The following are simple definitions for FPIR and FNIR:

A threshold "T" is used to classify a test case to be either a correct (true or positive) case or false (negative) case. If the case is below a threshold "T" then it is classified false (negative) and if it is above threshold "T" it is classified true (positive).

FPIR: is the false positive identification rate. It is the ratio of the test cases that are classified as true cases although they are false cases. It is type I error. 

FPIR= Number of test cases classified above threshold "T" (true) / Number of all test cases

FNIR: is the false negative identification rate. It is  the ratio of the test cases that are classified as false cases although they are true cases. It is type II error.

FNIR= Number of test cases classified below threshold "T" (false) / Number of all test cases

https://cran.r-project.org/web/packages/GGIR/index.html is your best bet. It is not developed specifically for Actigraph. However, it's relatively brand independent (http://www.ncbi.nlm.nih.gov/pubmed/27183118 )

.

excellent question ; the methods i know are mainly of the resampling kind

a good starting point :

.

Hello David et alii,

Seen as a theoretical formula : " How to reduce a list of factors ..." could sound as thinking to build general types of computing as pattern processes in a nearly neuronal way. But you introduce the question in a more analytical manner when we admit that " reducing a list " could not only be the economical part of the work but also  finding a processing solution.

Practically I remember about LISP a language thougnt for manipulating lists you could be interrested in. Could a neural network process achieve the task as you wish?

You can also ask your question to people interrested in Set Operads related to combinatorics for example a book : Set Operads in Combinatorics and Computer Sciences. By Miguel A. Mendez. Springer briefs in mathematics. Springer.

But it seems to me aslo true that your question as a specific practical and direct problem could largely progress in your mind if better defined. and more deeply explanable for any further help.

Best wishes.

Jean-Yves Tallet

.

you might be interested in reading :

.

Fraud is a very general term and you need to be more specific on the type of fraud that you are targeting. I would suggest the following reference:

If you look at the preview it will show you the fraud tree which are the subcategories of fraud. Each of these types of fraud have particular characteristics that you can target. Some of them will be outlier detection through clustering, while others will be based on values within each vector for supervised learning.

 Thank you Sayed, but could you give me a hint on how can I get the sequential Alarm patterns from this alarm data?

Razieh -

It seems that perhaps your application here is for a univariate time series for which you have no other data to do anything more complex.  If your current value is "missing," then that sounds like you are really saying that you need a forecast.   And because you have no clearly defined patterns established in your time series, the simplest format for "exponential smoothing" may be as good as you can do, at least for now.  It will put the strongest influence on the most recent data that you do have. 

Jim 

Are you searching for digit databases or general ones?

For the latter I think the ImageNet DB contains more than enough data. ;)

There are several ways, some that you can use without using algorithms from machine learning/soft computing/etc. For example, you can use curvature (see attached graphic) and find the maximum curvature for curves within each of your classes and simply group curves based upon the similarity of the resulting values. Similarly, for a curve gamma we can obtain the value k of the curve at p=gamma(x) using the equation in the second attached image, if the curve is smooth you can obtain global curvature using the integral in the third attached image.

There are many other mathematical ways to classify curves, but I suspect you are using the word colloquially, making many of the mathematical approaches useless (i.e., you really mean a depiction of a line with one or more "bends" in it, not something like the Peano curve or Koch curve).

Also, mathematically curves are generally already classified. The simple classification schemes may be similar to what you used to form your initial 3 classes (e.g., parabolas, hyperbolas, ellipses, etc.).

Then there are statistical analyses of shape, such as flux graphs using AOFs (average ouward flux) or other medial representation based approaches (e.g., flux based skeletonization). I've attached a chapter that might have something useful for you.

Thank you. I am actually interested in differences among seasons but the data are highly correlated across seasons and I am not sure about using season as a fixed effect. 

Thanks wei chen....i got a solution using R Studio...it has inbuilt Kmeans  algorithm...that for your expert advice...will be in touch for future...

Dear James Peters,

Yes, there are many applications of convexity, or convex sets, which sounds very similar to Christopher Alexander's positive space, one of the 15 structural properties he developed in The Nature of Order: http://en.wikipedia.org/wiki/The_Nature_of_Order

People tend to perceive things from this perspective of convexity in order to see identifiable pieces of a whole. This perspective (focusing on individuals) however is somehow contradictory to the holistic (or fractal) perspective. For example, drop a wine glass on the ground, and it is very likely broken into many pieces. These many pieces are actually mixed up with convex and concave ones. This mixed up is the same for natural cities we identified or detected using massive geographic information; see related paper: http://arxiv.org/ftp/arxiv/papers/1501/1501.03046.pdf

Having said so, may I draw a tentative conclusion as such? The perspective of convexity is Euclidean, focusing on individual, relying on human perception, whereas the fractal or holistic perspective is not necessary to be convex only, but mixed up with convex and concave ones. For example, the math snowflake is concave in essence, and is made of many convex triangles of course. 

Dear Luca,

you were right with finding the statement suspicious. Simultaneously diagonalizable matrizes would indeed commute, and it is easy to see that this is not true in general, even if one of the matrizes is assumed to be positive definite. (For example take a diagonal 2x2 Matrix with entries 1 and 2 and the 2x2 matrix with all four entries equal to 1.) So two symmetric matrizes cannot be diagonalized simultaneously in general.

The book by Fukunaga you mention indeed does something different. Namely, given a positive definite matrix X and a symmetric matrix Y, the author finds a (non-orthogonal) invertible matrix A such that A^tXA and A^tYA are both diagonal (so he uses the transpose rather than the inverse) and indeed A^tXA is the identity matrix. So this does not actually concern diagonalization of symmetric matrizes but of bilinear forms. The result can be expressed invariantly as the fact that given two symmetric bilinear forms, one of which is positive definite, there is a basis which is orthogonal for both forms.  This is not a deep result, it can be deduced from standard linear algebra results as follows: There is an orthonormal basis for the positive definite form. The other form is represented with respect to this basis by a symmetric matrix and then the usual oprthogonal diagonalization of symmetric matrices gives you an orthonormal basis for the first form with respect to which the second form is diagonal. (The book uses this argument converted to matrix form.)

hello,

you should read "feature selection methods" like filters and wrappers

and hybrid approaches.

I recommend you this papers;

"A survey on feature selection methods"   

Girish Chandrashekar , Ferat Sahin

Journal Computers and Electrical Engineering 40 (2014) 16–28

and

"A novel feature selection approach for biomedical data classification"

Yonghong Peng , Zhiqing Wu, Jianmin Jiang

Journal of Biomedical Informatics 43 (2010) 15–23

Best Regards

The first decision is if you're trying to find a pattern that you know something about (supervised case), or try find a pattern without knowing anything about it (unsupervised case).

I don't have much experience in the unsupervised task, which is what interests you probably.

I would start my search by combining terms like "time series" and "motif discovery".

A quick search gives these papers that seem like they might be what you're looking for:

http://www.sciencedirect.com/science/article/pii/S0952197610001727 ("Mining in time series section")

I don't have enough experience with these types of tools or your type of data to give you a single recommendation.

A few additional references for your consideration:

Maye, J., Weiss, D. J., & Aslin, R. N. (2008). Statistical phonetic learning in infants: Facilitation and feature generalization. Developmental Science, 11(1), 122-134.

Keidel, J. L., Jenison, R. L., Kluender, K. R., & Seidenberg, M. S. (2007). Does grammar constrain statistical learning? Commentary on Bonatti, Pena, Nespor, and Mehler (2005). Psychological Science, 18(10), 922-923.

ten Cate, C., & Okanoya, K. Revisiting the syntactic abilities of non-human animals: natural vocalizations and artificial grammar learning. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1598), 1984-1994.

Hi Osama, sorry for the slow reply. These files do indeed appear to be NIfTI-2 (the header size is 540, as described here http://nifti.nimh.nih.gov/pub/dist/doc/nifti2.h, which you can check with f=fopen('file'); fread(f,1,'int') in matlab).

SPM12b can read them, though it warns that "code 4 is not an option for units". It seems that the files are one-dimensional (i.e. not images, but perhaps individual time-series), with length 91282. E.g. the following works for me in SPM12b:

 N = nifti('cope1.dtseries.nii')

 N.dat.dim % [1 1 1 1 91282]

 plot(squeeze(N.dat(1,1,1,1,:)))

I've checked that SPM12b is happy with one of the NIfTI-2 test images from here: http://nifti.nimh.nih.gov/pub/dist/data/nifti2/ and it does seem to be, so if you were not expecting these files to be 1D, then you might need to investigate further in the connectome workbench (I can't help you with that I'm afraid).

I think the previous two references provided by @Christos P Loizou and @Lucia Ballerini are new and excellent to start with.

Thanks Hasan, but what do you mean of the camera position, orientation, focal length? I think it will be the same for all image.

Or you can try this

for i=1 to n

for j=1:n

f= p(i,j) log p(i,j)

end

end

bwlabel is used on a binary image and it gives the number of connected objects/components in a binary image.Using that label you can access the particular object and calculated its different features like area,centroid,eccentricity etc (use regionprops to calculate these) that can help in classification.Regarding label vector matrix i did not get what you mean by it.

As the people earlier have correctly pointed out, using K-means you can "segment" the image into K regions. However, you need to manually decide on K so that you have K distinct labelsl.

If you want to label the regions of a gray-image similar to bwlabel, you need to have a criteria of similarity between adjacent pixel values to have a region-level criteria. Please go through the function "imregionalmax" which finds a region level maxima from a gray-scale image, and works by the neighborhood relationship exactly similar to bwlabel.

You can also achieve the same through any region-growing segmentation algorithm, or even watershed segmentation algorithm.

Thank you

Late answer, but perhaps of interest for later readers; the similarity metric proposed by Warren et al. (Warren, D. L., Glor, R. E., & Turelli, M. 2008. Environmental Niche Equivalency Versus Conservatism: Quantitative Approaches to Niche Evolution. Evolution 62: 2868–2883.). It seems similar to the before mentioned root mean square deviation and can be used as a relative measure of similarity of the grids. To test for significance, you could perhaps create confidence intervals around the similarity index using bootstrapping.

In R you can calculate the Warren similarity index using the Istat function in the SDMTools package or the niche.overlap function in the phyloclim package. If you use Maxent for spatial distribution modeling, the hypothesis.testing function in the phyloclim package may be of interest.

In GRASS GIS I wrote an extension that calculates Warren's similarity: http://grasswiki.osgeo.org/wiki/AddOns/GRASS_6#r.niche.similarity

Well theoretically, it does find the optimal vector in the feature space such that two data sets are well separated while "maintaining a small variance for each group."

The line that you are talking about is the separating line or the hyperplane on which the projections are made? The hyperplane will definately be optimal.

Hi, try also looking through Numerical Recipes in C++. Although the algorithms are in C++, the background theory and examples are given in general. http://www2.units.it/ipl/students_area/imm2/files/Numerical_Recipes.pdf

Thank you

Hi Sajjad,

as Eric pointed out, the idea of using GMM for anomaly detection, OCC, etc. is not new.

Usually, one uses the estimated probability density p(x)=sum_i p(i)*p(x|i), i.e. the linear combination of cluster likelihoods you mentioned in your question above.

.

a small vocabulary problem here (maybe) :

in your example, the number of "features" (that is the dimensionality of the representation space of your data) is constant and equal to one for the three test sets. What is changed is the number of "instances" of the test sets, that is the number of elements ("cardinality") of the test sets.

now, i am not familiar with your matlab code but i assume that sum(lp) is the loglikelihood, that is the log probability of observing your test set given the model you buit on your training set.

Indeed, under the independence hypothesis for your observations in the test set, the probability is just the product of the probabilities of observing each sample (therefore the sum(lp) stuff) ; since a probability is always less than one, the more elements you have in your test set, the smaller the product of probabilities is.

As simple as that, assuming i understood your question correctly !

.

Are these 2 time series, i.e. 2 y-values for each x-value? In your example the data separates completely (the blue line is higher than the red line), so just taking the minimum of the maximums (=a) for each x-pair and then the max of the mins (=b) you'll have your "classifier" as (a+b)/2.

I suspect you have data where the 2 lines "mix". In that case, just a thought, may be try to model as 2 time series and use the models to separate the data?