Information Theory

Follow this topic

Answer added to:
43 Is it possible to measure the Quality of Information? And by the way - What is Information? And what is Quality?
By András Györfi · Hospital of Marghita

Doug Elias · Ithaca College

The problem, Andras, is that I don't know how to combine *my* 'Qi' with someone else's 'Qi', given that they're all unique, in such a way that the mer... [more]

The problem, Andras, is that I don't know how to combine *my* 'Qi' with someone else's 'Qi', given that they're all unique, in such a way that the merge will do what either of us expect it (or at least our contribution to it) to do ... especially when, by combining it with someone else's, I'm likely going to end up with a *lower* median 'Qi' (I can't say 'mean', because I'm still not convinced I understand how this ... thing ... is measured, or evaluated ... or *used*!). There has to be a calculus for this thing, Andras ... a training manual, a tutorial, SOMEthing to help us figure out what buttons it has and how to make the funny lights spin and flash and the OOOGGGGAAA horn to sound off. I'm still not convinced that 'Qi' acts as you think it does ... for example, does the fact that Newton wrote his Principia 400-500 years ago make it any less awesome today, even though we've got over 100 years of Special Relativity under our collective belts by now? After all, as long as our relative acceleration ratio to c remains very close to 0, Newton's Laws are an acceptable generalization. So, please show me how this works: how do you measure it, how do you utilize it, how do you make sure that you *emit* it? Doug

Following
Answer added to:
1 Are the Renyi and Havdra-Charvat (aka Tsallis) entropies real entropies?
By Bernard Lavenda · Università degli Studi di Camerino

Zolacir Oliveira Jr. · Universidade Estadual de Santa Cruz

No.

No.

Following
Answer added to:
2 I want to calculate the variance of the noise in a nonlinear channel. how do I do?
By Salah Bouhoun · École nationale polytechnique d'Alger

Yuriy Shmaliy · Universidad de Guanajuato

That is true for the variance in a linear channel. Why did you mention that the channel is nonlinear?

That is true for the variance in a linear channel. Why did you mention that the channel is nonlinear?

Following
Answer added to:
4 Channel Capacity?
By Manoj Gowda · Indian Institute of Technology Kanpur

Vahid Meghdadi · Université de Limoges

It is clear that when SNR -> infinity, you can use QAM-x where x->infinity. So for one channel use you can send infinity of bits without error (becaus... [more]

It is clear that when SNR -> infinity, you can use QAM-x where x->infinity. So for one channel use you can send infinity of bits without error (because SNR is large or noise -> 0). Therefore it is normal that with SNR->inf, C-> inf.

Following
Answer added to:
10 Information contained in a sampled signal
By Manoj Gowda · Indian Institute of Technology Kanpur

Jean-Paul Cipria · Nanotechinnov

Manoj, The sampling theorem says that you may reconstitute entirely a signal if you upper sample the signal and if the signal is sufficiently periodi... [more]

Manoj, The sampling theorem says that you may reconstitute entirely a signal if you upper sample the signal and if the signal is sufficiently periodic (not available for distribution). Entropie is an informations amount evaluation the signal carries. For example, if you under sample the signal then you lost information and the signal entropy decrease. h(i)=pi.log(1/pi) decrease because : pi statistically decrease or increase away from the maximum of h(i) / pi curve. But you can continue to use your signal. It is like you have noise on the signal. Sampling theorem is just a limit as : ------------------------------------------------- Low Bit rate but low efficienty <= Sampling Theorem = High efficienty rate <= High bit rate but low efficienty This type of theorem just show us how to raise efficienty up to n = 1. If you under sample then n<<1. If you upper sample then n<<1. If you are close to the theorem then n=0.99 for example. Best regards. Jean-Paul Cipria http://www.nanotechinnov.com/

Following
Answer added to:
2 Multiple access techniques under research
By Bashar Mufid · Al-Quds University

Following
Answer added to:
4 Who denoted first the entropy function by an H?
By Patrick Solé · ParisTech - Paris Institute of Technology

Josip Stepanic · University of Zagreb

Regarding only the notation, traces go back to L. Boltzmann, http://en.wikipedia.org/wiki/H-theorem.

Regarding only the notation, traces go back to L. Boltzmann, http://en.wikipedia.org/wiki/H-theorem.

Following
Answer added to:
8 Can anybody help me with conditional cross entropy or conditional mutual information computation?
By Sandipan Karmakar · IIT Kharagpur

Sandipan Karmakar · IIT Kharagpur

Thanks Dr Geng...

Thanks Dr Geng...

Following
Question:
Open How do I calculate the individual probabilities from an estimated kernel distribution?
I have my data set. I have obtained the different bandwidths (h_opt) but how do I actually compute the individual probabilities?

I have my data set. I have obtained the different bandwidths (h_opt) but how do I actually compute the individual probabilities?

By Zephyr Gola · University of Mauritius

Following
Answer added to:
4 Any topics ?
By Charlotte Hucher · Alcatel Lucent

Patrick Solé · ParisTech - Paris Institute of Technology

Is algebraic decoding dead? At least in applications?

Is algebraic decoding dead? At least in applications?

Following
Question:
Open Is it possible to find an information source with memory and infinite order ?
Infinite order = without order?

Infinite order = without order?

By Gheorghe Zaharia · Institut National des Sciences Appliquées de Rennes

Following
Answer added to:
4 Does the Holographic Principle contradict the Simulated Reality hypothesis?
By Gustavo Valdiviesso · Universidade Federal de Alfenas

Gustavo Valdiviesso · Universidade Federal de Alfenas

You are right again in your statement: A 2D surface cannot represent a 3D volume (I believe you meant volume instead of surface here, please correct m... [more]

You are right again in your statement: A 2D surface cannot represent a 3D volume (I believe you meant volume instead of surface here, please correct me if not). And that is the beauty behind the so called "holographic principle": It states that the amount of information stored in the degrees of freedom of a quantum state has a density limit. You can store as much information as it fits on any quantum state alone, but when dealing with a ensemble of states, the total amount of information stored in a given volume cannot exceed the amount of information that can be stored on the states belonging to the boundaries of such volume. As a consequence, matter itself has a maximum volumetric density, since every quantum state has to store its own quantum numbers as information. Starting from this hypothesis, one arrives at the Schwarzschild radius of a blackhole as the connection between the minimal surface and the minimal volume that a certain mass can occupy.

Following
Answer added to:
8 What is the interest of a linear code over a field of 13 elements ?
By Georges Ferdinand Randriafanomezantsoa Radohery · African Institute for Mathematical Sciences

Georges Ferdinand Randriafanomezantsoa Radohery · African Institute for Mathematical Sciences

@ Patrick Solé. Indeed she is in Cape Town if I'm not wrong, but I'm rather working with her collaborator the Professor Jamshid Moori.

@ Patrick Solé. Indeed she is in Cape Town if I'm not wrong, but I'm rather working with her collaborator the Professor Jamshid Moori.

Following
Answer added to:
15 How can we calculate the shannon entropy for a dataset which we want to train a machine with?
By Pouya Sinaian · Blekinge Institute of Technology

Pouya Sinaian · Blekinge Institute of Technology

@Michel Besserve: thanks alot . they seems to be helpful. Thanks

@Michel Besserve: thanks alot . they seems to be helpful. Thanks

Following
Answer added to:
9 Looking for a suitable error correcting code.
By Kimmo Halunen · VTT Technical Research Centre of Finland

Boris Assanovich · Yanka Kupala State University of Grodno

It's really nontrivial case to correct so great percent of errors. The codes proposed by people (LDPC and Turbo) which are very close to channel capa... [more]

It's really nontrivial case to correct so great percent of errors. The codes proposed by people (LDPC and Turbo) which are very close to channel capacity probably will not solve your problem for BER, SER or FER (?) you wrote. I think you have to make a deep look inside the problem considering the Source, Channel and result you want to get. There some approaches to Channel models, Joint-Source channel coding as well as Soure-Supported channel coding, etc.

Following
Question:
Open Which is the optimal coding technique for information source with memory ?
Which is the optimal coding technique for information source with memory ?

Which is the optimal coding technique for information source with memory ?

By Gheorghe Zaharia · Institut National des Sciences Appliquées de Rennes

Following
Answer added to:
7 Huffman Coding
By Gheorghe Zaharia · Institut National des Sciences Appliquées de Rennes

Following
Answer added to:
2 Mutual Information - Iterative Systems
By Hossein Mamani · Tarbiat Modares University

Following
Answer added to:
7 Do we need a new approach to communication in information theory ?
By Charlotte Hucher · Alcatel Lucent

Following

About Information Theory

An interdisciplinary study dealing with the transmission of messages or signals, or the communication of information. Information theory does not directly deal with meaning or content, but with physical representations that have meaning or content. It overlaps considerably with communication theory and CYBERNETICS.

Topic Followers (2906) See all

William Leibzon

University of California, Irvine
Guirec Lorant

France Télécom
Samuel Akinyede

University of South Wales
Karol Piczak

Warsaw University of Technology
Francesco Pantisano

University of Oulu
Xiaoan Huang

Shanghai Jiao Tong University
Nashwan mohammed al samman

Universiti Teknologi Malaysia
Yong Li

Chongqing University of Posts and Telecommunications
Rony Hermawan

Politeknik Elektronika Negeri Surabaya
Krystyna Maria

Gdynia Maritime University

Information Theory

About Information Theory

Topic Followers (2906) See all

Log in