Fig 21 - uploaded by Tommaso Melodia
Content may be subject to copyright.
Source publication
This article presents the design of a networked system for joint compression, rate control and error correction of video over wireless multimedia sensor networks (WMSNs) based on the theory of compressed sensing. First, compressed sensing based video encoding for transmission over WMSNs is studied. It is shown that compressed sensing can be used to...
Contexts in source publication
Context 1
... states that the derivative with respect to the sampling rate should be equal to a scaled version of the delay. Since U s ( γ I,s ) (as defined in (7)) is a concave monotonically increasing function in γ I,s , dU dγ s ( I,s γ I,s ) is decreasing in γ I,s . Therefore, as λ s varies, the optimal update direction of γ I,s is the negative of the direction of the change in round trip time . The simplest interpretation of ∆ RT T i ( t + 1) as calculated in (6) and used in (9) for source i is the difference between consecutive delay measurements λ i ( t ) − λ i ( t − 1) . The update direction of γ I,s is then given by ( − ∆ RT T ) , which is the direction of the update in (9). Finally, it was shown in [63] that given a small enough step size, a gradient projection algorithm such as (9) will converge to the optimal sampling rate allocation. Numerical simulations were also run to support this interpretation. Two simple networks were tested as shown in Fig. 20 and Fig. 21, respectively, where C i represents the capacity on link i and N i represents node i . The arrows represent video streams. In both cases, the optimal rate allocation was determined by solving the optimization problem directly as a semidefinite program using SeDuMi [61] with the convex optimization toolbox CVX [60], and the same problem was solved using the iterative algorithm (9). These two topologies were chosen because they verify two important requirements for a distortion based rate controller. The network in Fig. 20 has two video streams with a single bottleneck link. This topology can be used to assure that two different videos with different rate-distortion properties achieve the same received video quality. The other topology, shown in Fig. 21, was used to show that the rate controller will take advantage of unused capacity. Video 3 in this network is only contending with a single other video, while the other three videos are contending with each other resulting in a higher optimal rate for video 3. The results from these tests are shown in Figures 22, 23, 24 and 25. Figures 22 and 23 show the I frame sampling rate of the videos compared to the optimal value, and Fig. 24 and 25 show the actual video qualities. In all cases, the rate found by the iterative algorithm was within 5% of the optimal value as determined by the convex solver. The 5% difference between the optimal rates and the rates obtained from the iterative algorithm are due to the step size of the algorithm. If the step size were decreased, the resulting rate would be closer to the optimal. However, making the step size too small results in an algorithm which is infeasible to implement because of the amount of updates needed. Finally, to avoid the trivial solution of all rates and qualities being equal, different videos were transmitted. The simulations show that the iterative algorithm achieved all requirements, and was nearly optimal for both networks. IX. C ONCLUSIONS AND F UTURE W ORK This paper introduced a new wireless video transmission system based on compressed sensing. The system consists of a video encoder, distributed rate controller, and an adaptive parity channel encoding scheme that take advantage of the properties of compressed sensed video to provide high-quality video to the receiver using a low-complexity video sensor node. The rate controller was then shown to be an implementation of an iterative gradient descent solution to the optimal rate allocation optimization problem. Simulation results show that the C-DMRC system results in a 5%-10% higher received video quality in both a network with a higher load and a small load. Simulation results also show that fairness is not sacrificed, and is in fact increased, with the proposed system. Finally, the video encoder, adaptive parity and rate controller were implemented on a USRP2 software defined ratio. It was shown that the rate controller correctly reacts to congestion in the network based on measured round trip times, and that the system works over real channels. We intend to implement the remaining portions of the C-DMRC system on the USRP2 radios, including image capture and video decoding. We will also measure the performance and complexity of this system compared to state-of-the-art video encoders (H.264, JPEG-XR, MJPEG, MPEG), transport (TCP, TFRC) and channel coding (RCPC, Turbo ...
Context 2
... states that the derivative with respect to the sampling rate should be equal to a scaled version of the delay. Since U s ( γ I,s ) (as defined in (7)) is a concave monotonically increasing function in γ I,s , dU dγ s ( I,s γ I,s ) is decreasing in γ I,s . Therefore, as λ s varies, the optimal update direction of γ I,s is the negative of the direction of the change in round trip time . The simplest interpretation of ∆ RT T i ( t + 1) as calculated in (6) and used in (9) for source i is the difference between consecutive delay measurements λ i ( t ) − λ i ( t − 1) . The update direction of γ I,s is then given by ( − ∆ RT T ) , which is the direction of the update in (9). Finally, it was shown in [63] that given a small enough step size, a gradient projection algorithm such as (9) will converge to the optimal sampling rate allocation. Numerical simulations were also run to support this interpretation. Two simple networks were tested as shown in Fig. 20 and Fig. 21, respectively, where C i represents the capacity on link i and N i represents node i . The arrows represent video streams. In both cases, the optimal rate allocation was determined by solving the optimization problem directly as a semidefinite program using SeDuMi [61] with the convex optimization toolbox CVX [60], and the same problem was solved using the iterative algorithm (9). These two topologies were chosen because they verify two important requirements for a distortion based rate controller. The network in Fig. 20 has two video streams with a single bottleneck link. This topology can be used to assure that two different videos with different rate-distortion properties achieve the same received video quality. The other topology, shown in Fig. 21, was used to show that the rate controller will take advantage of unused capacity. Video 3 in this network is only contending with a single other video, while the other three videos are contending with each other resulting in a higher optimal rate for video 3. The results from these tests are shown in Figures 22, 23, 24 and 25. Figures 22 and 23 show the I frame sampling rate of the videos compared to the optimal value, and Fig. 24 and 25 show the actual video qualities. In all cases, the rate found by the iterative algorithm was within 5% of the optimal value as determined by the convex solver. The 5% difference between the optimal rates and the rates obtained from the iterative algorithm are due to the step size of the algorithm. If the step size were decreased, the resulting rate would be closer to the optimal. However, making the step size too small results in an algorithm which is infeasible to implement because of the amount of updates needed. Finally, to avoid the trivial solution of all rates and qualities being equal, different videos were transmitted. The simulations show that the iterative algorithm achieved all requirements, and was nearly optimal for both networks. IX. C ONCLUSIONS AND F UTURE W ORK This paper introduced a new wireless video transmission system based on compressed sensing. The system consists of a video encoder, distributed rate controller, and an adaptive parity channel encoding scheme that take advantage of the properties of compressed sensed video to provide high-quality video to the receiver using a low-complexity video sensor node. The rate controller was then shown to be an implementation of an iterative gradient descent solution to the optimal rate allocation optimization problem. Simulation results show that the C-DMRC system results in a 5%-10% higher received video quality in both a network with a higher load and a small load. Simulation results also show that fairness is not sacrificed, and is in fact increased, with the proposed system. Finally, the video encoder, adaptive parity and rate controller were implemented on a USRP2 software defined ratio. It was shown that the rate controller correctly reacts to congestion in the network based on measured round trip times, and that the system works over real channels. We intend to implement the remaining portions of the C-DMRC system on the USRP2 radios, including image capture and video decoding. We will also measure the performance and complexity of this system compared to state-of-the-art video encoders (H.264, JPEG-XR, MJPEG, MPEG), transport (TCP, TFRC) and channel coding (RCPC, Turbo ...
Similar publications
Compressed-sensing reconstruction of still images and video sequences driven by multihypothesis predictions is considered. Specifically, for still images, multiple predictions drawn for an image block are made from spatially surrounding blocks within an initial non-predicted reconstruction. For video, multihypothesis predictions of the current fram...
Citations
... The data with reduced dimensions is then used for classical machine learning (ML)-based readout. [20][21][22][23] , these methods often struggle with maintaining data integrity during the reduction process, leading to loss of critical information 24 . Our findings demonstrate that qPCA outperforms classical PCA (cPCA) in preserving critical information during dimensionality reduction, leading to more efficient and reliable data modeling. ...
The rapid growth of Internet of Things (IoT) devices necessitates efficient data compression techniques to handle the vast amounts of data generated by these devices. In this context, chemiresistive sensor arrays (CSAs), a simple-to-fabricate but crucial component in IoT systems, generate large volumes of data due to their simultaneous multi-sensor operations. Classical principal component analysis (cPCA) methods, a common solution to the data compression challenge, face limitations in preserving critical information during dimensionality reduction. In this study, we present quantum principal component analysis (qPCA) as a superior alternative to enhance information retention. Our findings demonstrate that qPCA outperforms cPCA in various back-end machine-learning modeling tasks, particularly in low-dimensional scenarios when limited Quantum bits (qubits) can be accessed. These results underscore the potential of noisy intermediate-scale quantum (NISQ) computers, despite current qubit limitations, to revolutionize data processing in real-world IoT applications, particularly in enhancing the efficiency and reliability of CSA data compression and readout.
... However, the large data transfers for WMSN make energy conservation the greatest tool. Compressed Sensing (CS) was introduced by Pudlewski et al. [4] as a means to overcome this. The CS mechanism is appropriate for WMSN owing to its low complexity, high compression rate, and robustness to transmission errors [5]. ...
A measurement matrix is essential to compressed sensing frameworks. The measurement matrix can establish the fidelity of a compressed signal, reduce the sampling rate demand, and enhance the stability and performance of the recovery algorithm. Choosing a suitable measurement matrix for Wireless Multimedia Sensor Networks (WMSNs) is demanding because there is a sensitive weighing of energy efficiency against image quality that must be performed. Many measurement matrices have been proposed to deliver low computational complexity or high image quality, but only some have achieved both, and even fewer have been proven beyond doubt. A Deterministic Partial Canonical Identity (DPCI) matrix is proposed that has the lowest sensing complexity of the leading energy-efficient sensing matrices while offering better image quality than the Gaussian measurement matrix. The simplest sensing matrix is the basis of the proposed matrix, where random numbers were replaced with a chaotic sequence, and the random permutation was replaced with random sample positions. The novel construction significantly reduces the computational complexity as well time complexity of the sensing matrix. The DPCI has lower recovery accuracy than other deterministic measurement matrices such as the Binary Permuted Block Diagonal (BPBD) and Deterministic Binary Block Diagonal (DBBD) but offers a lower construction cost than the BPBD and lower sensing cost than the DBBD. This matrix offers the best balance between energy efficiency and image quality for energy-sensitive applications.
... Following the same formulation as [8], [13], we can then find the Lagrangian of (4), as: ...
E-commerce platforms like Amazon, Jumia, Airbnb, Alibaba, eBay and JD.com among others play a huge role in linking sellers of products to interested buyers. Consumers generally ask questions in order to know if the product of interest will meet their needs. Relying on existing product reviews posted online by other consumers who have once purchased and used the product becomes a viable option to waiting for direct answers from the community question answer system. They are usually forced to spend time manually wading through these numerous customer reviews, a task which is cumbersome. This motivated to use Bi-directional Auto-regressive Transformer (BART) model for both question-answerability classification and answer generation. A dataset provided by Gupta et al and conducted on Amazon Mechanical Turk was used for the experiment. It was observed that BART outperformed the heuristic baselines on all the metrics considered with existing models of long short term memory, convolutional neural network and bidirectional encoder representative from transformers. Based on the BART performance, a fuzzy based rule was introduced with system parameters defined for the classification of product-related question answering review into linguistic variables for good purchase decision making. The classification is in line with well-formed responses to product-related questions based on existing customer reviews using BART heuristic baselines.
... In Mahalanobis and Muise, 22 compressed sensing is applied for sensors that can reconstruct images completely using relatively simple hardware for surveillance applications. In Pudlewski et al., 23 a cross-layer system is suggested for maximizing the captured video quality by controlling the channel coding rate, the transmission rate, and also the video encoding rate in wireless multimedia sensor networks (WMSNs). It is also presented that the compressed sensing sampling rate can control the rate of the compressed sensed video. ...
Wireless visual sensor networks (WVSN) have vital roles in surveillance applications. In these networks, wireless visual sensors include camera and transceiver module and collect visual information. However, energy consumption and coverage of the tracked targets are important challenges in WVSNs since increasing the coverage leads to increasing energy consumption. Therefore, energy optimization and satisfying the quality of experience (QoE) of the tracked targets are essential issues in these networks. In fact, the appropriate focal length setting leads to increasing the target coverage and quality of the captured targets' images and energy consumption. Therefore, selection of the suitable visual sensors and setting their focal length can overcome the energy consumption challenges and improve QoE of the tracked targets. In this case, compressive sensing is also expected to overcome the battery constraints of the WVSN resources. In this paper, the problem is to minimize the energy consumption of the multi‐target tracking with high reliability, while the coverage and also the quality of the received image of the targets are satisfied by selection of the proper visual sensors and the focal length adjustment. The convex optimization method is used to solve the problem. Also, based on the Karush‐Kuhn‐Tucker conditions, the optimal solution for the problem is obtained. Simulation results validate the efficiency of the proposed method in comparison with the other bench mark algorithms.
... Wireless Multimedia Sensor Networks (WMSN) enable innovative video surveillance applications [1]. However, WMSN work in energy-constrained environments which require novel data compression to reduce the transmission bandwidth and computational complexity [2]. ...
... However, WMSN work in energy-constrained environments which require novel data compression to reduce the transmission bandwidth and computational complexity [2]. Compressed Sensing (CS) was proposed by Pudlewski et al. [1] as a tool to solve this problem. The CS approach is well suited for WMSN because of diminished complexity, substantial compression rate and high resilience to channel errors [3]. ...
Wireless Multimedia Sensor Networks (WMSN) hold the key to unlocking the next generation of video surveillance applications. They operate under energy-constrained environments but Compressive Sensing (CS) is a tool that can help overcome these challenges. Sensing matrices are critical in delivering the promise of CS, there are different types and each has its benefits and costs. In this paper, these sensing matrices are compared and the strengths and weaknesses were highlighted. It was found that deterministic sensing matrices held the most promise as they gave better recovery accuracy than dense random matrices while being more efficient but, work still needs to be done to evaluate the energy cost of their implementation.
... Wireless Multimedia Sensor Networks (WMSNs) are systems of embedded devices deployed to fetch, process and collate multimedia streams from dissimilar sources [1]. WMSNs allow new applications such as video surveillance, storage and recovery of actions and locations of people [2]. This entails advanced data compression for lessening the bandwidth and energy utilisation of the sensor nodes [3]. ...
... Encoder complexity and poor resilience to channel errors are the two important limitations of systems based on the transmission of predictively encoded video through a layered wireless communication protocol stack [4]. Compressed Sensing (CS) was proposed by Pudlewski et al. [2] as the solution to overcoming these challenges. CS allows for under-sampling of sparse signals through an encoder with little complexity. ...
... There are numerous integrated devices in wireless sensor networks, such as low-cost sensors, a microphone to get multimedia information from the area, such as video streams, audio streams, and scalar sensor data. In Initially, Wireless Sensor Networks (WSNs) that gather the dispersed information, and edge computing enables the effectiveness of conventional sensor networks [7]. Wireless multimedia sensor networks are a new type of network integrating video, audio, images, and other multimedia processing functions that are based on conventional Wireless Sensor Networks (WSNs) [8,9]. ...
In recent years, the infrastructure of Wireless Internet of Sensor Networks (WIoSNs) has been more complicated owing to developments in the internet and devices' connectivity. To effectively prepare, control, hold and optimize wireless sensor networks, a better assessment needs to be conducted. The field of artificial intelligence has made a great deal of progress with deep learning systems and these techniques have been used for data analysis. This study investigates the methodology of Real Time Sequential Deep Extreme Learning Machine (RTS-DELM) implemented to wireless Internet of Things (IoT) enabled sensor networks for the detection of any intrusion activity. Data fusion is a well-known methodology that can be beneficial for the improvement of data accuracy, as well as for the maximizing of wireless sensor networks lifespan. We also suggested an approach that not only makes the casting of parallel data fusion network but also render their computations more effective. By using the Real Time Sequential Deep Extreme Learning Machine (RTS-DELM) methodology, an excessive degree of reliability with a minimal error rate of any intrusion activity in wireless sensor networks is accomplished. Simulation results show that wireless sensor networks are optimized effectively to monitor and detect any malicious or intrusion activity through this proposed approach. Eventually, threats and a more general outlook are explored.
... A cross-layer technique for streaming videos was designed in WMSN in Pudlewski et al. [11], using the concept of Compressive Sensor, aimed at altering data transmission speeds. The broadband needs for sensing nodes can be spectacular for all if many network source streams premium video. ...
Wireless Multimedia Sensor Networks (WMSN) provides valuable information for scalar data, images, audio, and video processing in monitoring and surveillance applications. Multimedia streaming, however, is highly challenging for networks as energy restriction sensor nodes limit the potential data transmission bandwidth and lead to reduced throughput. WMSN's two key design challenges, which can be achieved by the clustering process, are energy efficiency and throughput maximization. The use of the clustering technique helps to orga-nise the sensor nodes into clusters, and between each cluster a cluster head (CH) will be chosen. This paper introduces a new Artificial Fish Swarm Optimization Algorithm (AFSA) with a Clustering Technique for Throughput Maximization in WMSN, called AFSA-HC, based on Hill Climbing (HC). The proposed AFSA-HC algorithm includes four key processes to optimise network throughput, namely node initialization, node clustering based on AFSA-HC, data aggregation based on the deflate algorithm, and transmission of hybrid data. To check the adequate performance of the presented AFSA-HC technique, a thorough experimental review will be carried out. The results of the simulation showed that the AFSA-HC approach achieved optimum results for various steps, namely energy consumption, throughput, network life, network stability and packet loss.
... Among the different metrics, the similarity index is considered as a good objective metric due to its proven performance [32]. The state of the art shows that this metric has also been selected as a metric to assess the QoE (see e.g., [47][48][49][50]). ...
... In [49], the authors used the SSIM to measure the quality of video transmission through their system (Compressive Distortion Minimizing Rate Control, C-DMRC). The latter uses a distributed cross-layer control algorithm that aims to maximize the received video quality over a multi-hop wireless network with lossy links. ...
The DASH (Dynamic Adaptive Streaming over HTTP) standard is widely adopted for video streaming. The Adaptive BitRate (ABR) style adaptation mechanism, which is a key component of DASH, is not standardized, since it must take various elements into account, in particular the context of the communication and the system, but also the quality perceived by the users, to maximize the QoE (Quality of Experience). Many ABR algorithms have been proposed. Few of them attach importance to perceived, and objectively calculated, quality as an adaptation parameter. This thesis proposes a generic framework, called VQBA (Video Quality Metric Based Adaptation algorithm), allowing to integrate an objective metric of the video quality of one’s choice as an adaptation parameter. The idea is to maximize the eÿcient use of the available bandwidth by deciding to switch to a higher speed not only because it is feasible, but also because it provides a significant visual improvement. We carried out numerous tests with video sequences of various kinds and by placing them in real network situations with traces from operational mobile networks. These tests, through three usual video quality metrics, namely SSIM (Structural Similarity Index Measurement), PSNR (Peak Signal to Noise Ratio) and VMAF (Video Multimethod Assessment Fusion), and in comparison with a selection of ABR algorithms, show that the path we explored, that is to say, giving importance to video quality as an adaptation parameter, is an e˙ective path for better QoE
... fused to attain the maximum QoS [12]. Non-hierarchical data exchange between various layers takes place in cross-layer based routing architecture [13]. ...
... Thus, the formulated objective function includes both the energy and distance of every path. The fitness function is formulated as the minimization function and it is the product of energy and distance, which is given in (12). F = min{RE × DIST } (12) where, 'F ' is the fitness of i th population, 'RE ' is the energy required in the i th population and 'DIST ' is the total distance of i th path or population. ...
... The fitness function is formulated as the minimization function and it is the product of energy and distance, which is given in (12). F = min{RE × DIST } (12) where, 'F ' is the fitness of i th population, 'RE ' is the energy required in the i th population and 'DIST ' is the total distance of i th path or population. ...
Nowadays, the Internet of Things (IoT) is adopted to enable effective and smooth communication among different networks. In some specific application, the Wireless Sensor Networks (WSN) are used in IoT to gather peculiar data without the interaction of human. The WSNs are self-organizing in nature, so it mostly prefer multi-hop data forwarding. Thus to achieve better communication, a cross-layer routing strategy is preferred. In the cross-layer routing strategy, the routing processed through three layers such as transport, data link, and physical layer. Even though effective communication achieved via a cross-layer routing strategy, energy is another constraint in WSN assisted IoT. Cluster-based communication is one of the most used strategies for effectively preserving energy in WSN routing. This paper proposes a Bio-inspired cross-layer routing (BiHCLR) protocol to achieve effective and energy preserving routing in WSN assisted IoT. Initially, the deployed sensor nodes are arranged in the form of a grid as per the grid-based routing strategy. Then to enable energy preservation in BiHCLR, the fuzzy logic approach is executed to select the Cluster Head (CH) for every cell of the grid. Then a hybrid bio-inspired algorithm is used to select the routing path. The hybrid algorithm combines moth search and Salp Swarm optimization techniques. The performance of the proposed BiHCLR is evaluated based on the Quality of Service (QoS) analysis in terms of Packet loss, error bit rate, transmission delay, lifetime of network, buffer occupancy and throughput. Then these performances are validated based on comparison with conventional routing strategies like Fuzzy-rule-based Energy Efficient Clustering and Immune-Inspired Routing (FEEC-IIR), Neuro-Fuzzy-Emperor Penguin Optimization (NF-1318 Tandon et al.: A Bio-inspired Hybrid Cross-Layer Routing Protocol for Energy Preservation in WSN-Assisted IoT EPO), Fuzzy Reinforcement Learning-based Data Gathering (FRLDG) and Hierarchical Energy Efficient Data gathering (HEED). Ultimately the performance of the proposed BiHCLR outperforms all other conventional techniques.
