ChapterPDF Available

Digital-Physical Parity for Food Fraud Detection

Authors:

Abstract and Figures

Food fraud has an adverse impact on all stakeholders in the food production and distribution process. Lack of transparency in food supply chains is a strong factor contributing to food fraud. With limited transparency, the insights on food supply chains are fragmented, and every participant has to rely on trusted third parties to assess food quality. Blockchain has been introduced to the food industry to enable transparency and visibility, but it can only protect the integrity of a digital representation of physical food, not the physical food directly. Tagging techniques, like barcodes and QR codes that are used to connect the physical food to its digital representation, are vulnerable to attacks. In this paper, we propose a blockchain-based solution to link physical items, like food, to their digital representations using physical attributes of the item. This solution is generic in its support for different methods to perform the physical checks; as a concrete example, we use machine learning models on visual features of food products, through regular and thermal photos. Furthermore, we use blockchain to introduce a reward system for supply chain participants, which incentivizes honesty and supplying data. We evaluate the technical feasibility of components of this architecture for food fraud detection using a real-world scenario, including machine-learning models for distinguishing between grain-fed and grass-fed beef.
Content may be subject to copyright.
Digital-Physical Parity for Food Fraud
Detection
Sin Kuang Lo1,2(B
),XiweiXu
1,2, Chen Wang1, Ingo Weber1,2, Paul Rimba1,
Qinghua Lu1,2, and Mark Staples1,2
1Data61, CSIRO, Sydney, Australia
{sinkuang.lo,xiwei.xu,chen.wang,ingo.weber,paul.rimba,
qinghua.lu,mark.staples}@data61.csiro.au
2School of Computer Science and Engineering, UNSW, Sydney, Australia
Abstract. Food fraud has an adverse impact on all stakeholders in the
food production and distribution process. Lack of transparency in food
supply chains is a strong factor contributing to food fraud. With lim-
ited transparency, the insights on food supply chains are fragmented,
and every participant has to rely on trusted third parties to assess food
quality. Blockchain has been introduced to the food industry to enable
transparency and visibility, but it can only protect the integrity of a
digital representation of physical food, not the physical food directly.
Tagging techniques, like barcodes and QR codes that are used to con-
nect the physical food to its digital representation, are vulnerable to
attacks. In this paper, we propose a blockchain-based solution to link
physical items, like food, to their digital representations using physical
attributes of the item. This solution is generic in its support for different
methods to perform the physical checks; as a concrete example, we use
machine learning models on visual features of food products, through
regular and thermal photos. Furthermore, we use blockchain to intro-
duce a reward system for supply chain participants, which incentivizes
honesty and supplying data. We evaluate the technical feasibility of com-
ponents of this architecture for food fraud detection using a real-world
scenario, including machine-learning models for distinguishing between
grain-fed and grass-fed beef.
Keywords: Blockchain ·Machine learning ·Food fraud
1 Introduction
Food fraud is a US$40 billion per year industry, which has a significant negative
impact on all stakeholders in the food production and distribution process. Food
fraudsters leverage consumers’ perceived safety, trust, and value associated with
prominent brands and certifications to charge premium prices for inferior, or even
dangerous, food products. However, it is difficult, if not impossible, to eliminate
the fraud industry considering the big profit it generates for beneficiaries.
c
Springer Nature Switzerland AG 2019
J. Joshi et al. (Eds.): ICBC 2019, LNCS 11521, pp. 65–79, 2019.
https://doi.org/10.1007/978-3-030-23404-1_5
66 S. K. Lo et al.
Lack of transparency and visibility of the food supply chain is a strong factor
contributing to food fraud. Food supply chains are complex multi-party systems
that involve different participants, such as farmers, food production processors,
and retailers. However, regulations in major markets, such as the US, the EU, and
China generally only require one-up and one-down traceability regarding supply
chain participants [1]. In a food supply chain with limited transparency, insights
are fragmented and thus each consumer or producer has to rely on trusted third
party agencies to monitor food quality across the whole food supply chain.
Blockchain technology has been applied to food supply chains [3,5] to enable
transparency and visibility because every participant within the blockchain net-
work has access to all the records and historical movements of information in
the entire system. For instance, a blockchain-based traceability system has been
proposed for tracking food products along international supply chains [8]. How-
ever, blockchain only guarantees the integrity of the digital representation of
food but not the integrity of the physical attributes of a food product.
Commonly used tagging technologies, like barcodes, serial numbers, and QR
codes can connect physical food to its digital representation. Such techniques
are vulnerable to attacks, e.g., a counterfeit product with authentic packaging
and tag1. To counter fraud, it is necessary to enable supply chain participants
to verify the connection between the food itself and the digital information.
Traceability Informaon
Food Product
Digital Representaon
Physical Aributes
Physical World (O-chain)
Physical label
Physical ID
(RFID,Barcode, QR code)
Describe
Blockchain (On-chain) a
Fig. 1. Conceptual overview of food fraud detection
A conceptual overview of the relation between a food product, its physical
label, ID, attributes and a digital representation is shown in Fig. 1. The integrity
we seek to establish is between the physical food product and its digital represen-
tation. We approach this goal by connecting the physical item via its label and
1https://www.news.com.au/finance/business/retail/empty-tin-sales-a-new-low-in-
baby-formula-saga/news-story/b269717700fa9128470803cb34363705.
Digital-Physical Parity for Food Fraud Detection 67
physical attributes to the digital system (links marked (a) in Fig.1). Conceptu-
ally, this architecture takes a similar approach as in multi-factor authentication
and biometric cryptosystems where physiological and behavioral features are
used for identifying a user [7]. The difference is mainly on that the physical
characteristics of a food product is rich and diverse, which is difficult to capture
using a closed system.
Stable isotope ratio analysis is one technique in this direction: chemical iso-
tope levels on physical food products, e.g., on muscles of cows [4], can be analyzed
to verify claims of origin. However, isotope analysis is not economically feasible
at large to tackle food fraud issues. Similarly, a national lab for drug substance
testing recently tested honey samples sourced from supermarkets and markets in
Australia; adulteration was detected in 18% of the 38 samples [10]. While indepen-
dent labs may provide test results, the confirmation of their discovery takes time.
Test methods need to be fast and cheap enough to enable widespread use.
We propose using machine learning techniques to assess food products
through exploiting distinct visual features to verify their physical attributes.
Machine learning models provide an effective way to encapsulate knowledge of
differentiating objects based on features. Data can be collected from cameras,
smartphones, X-ray, and ultrasound scanners. By training models and perform-
ing feature extraction across training data, we can verify the physical attributes
of a food product against its digital information. Recipients in the supply chain
can check food authenticity based on features they collect.
In this paper, we make the following contributions:
We propose a blockchain-based architecture that enables checking the parity
of physical items and their digital representation. The proposal is generic in
its support for methods to check physical attributes against the digital claims
about these attributes.
As one concrete method, we propose machine learning on distinct visual fea-
tures of food. In this method, we provide the exposure and knowledge about
food fraud to the consumers by machine learning models.
We devise incentive mechanisms that distribute coins to food supply chain
participants to reward honest behavior and supplying data. These rewards can
further be shared with machine learning developers, who contribute models
that allow fraud detection.
We evaluate the technical feasibility of components of this architecture for food
fraud detection using a real-world scenario, including machine-learning mod-
els for distinguishing between grain-fed and grass-fed beef. To the best of our
knowledge, there is no related work that use machine learning and blockchain to
link between the physical attributes of physical food products and their digital
representations.
The remainder of the paper is organized as follows. The overview of the
generic architecture is listed in Sect. 2, followed by the resulting system design
in Sect. 3. Section 4provides technical details of the system. The evaluation with
a beef case study is described in Sect. 5. Finally, Sect. 6concludes the paper and
outlines the future work.
68 S. K. Lo et al.
2 A Generic Architecture for Food Digtal-Physical Parity
2.1 Overview
The generic architecture for food digital-physical parity is shown in Fig.2.The
architecture aims to connect the physical items in a supply chain with their
digital representation. The attributes of a physical items (e.g., visual features,
geographical location and chemical composition) can be assessed via various
techniques. There is no perfect solution that fits all different kinds of food prod-
ucts: for some, it might be more suitable to check their visual attributes but
for others it might only be possible to check via their chemical composition.
Hence, our generic architecture allows the use of different types of assessment
methods to verify the claimed attributes on physical products. For the digital
representation of the physical items, blockchain is chosen due to its immutable
and transparent nature and ability to span across dynamic networks of parties.
Functionally, blockchain can be used as a data storage and a computational
infrastructure in a software system [9]. As a data storage, blockchain stores all
transactions that have ever occurred in the network, which cannot be deleted
or modified. As a computational infrastructure, blockchain can run programs
called smart contracts, the result of which is stored in the distributed trusted
data storage.
Fig. 2. A generic architecture for food digital-physical parity
The claimed attribute assessment methods requires computational resources
that can currently not be fulfilled by blockchain. Therefore, the computation of
the assessment is done off-chain. Data required for the execution of assessment
are also stored off-chain.
Information related to the food supply chain, such as the claimed attributes
of food products, the details about stakeholders on the supply chain are being
recorded on blockchain. The transactions on blockchain enables the recipients
Digital-Physical Parity for Food Fraud Detection 69
to trace the originality and the processing activities on the food product in the
corresponding supply chain. Incentive mechanism would be deployed as a smart
contract on the blockchain to motivate active participants to contribute to the
supply chain. The results from the assessment of the claimed attributes will be
used as the input for the incentive mechanism to determine the reward to the
contributors.
As blockchain is unable to communicate with the external world, an oracle
is required to enable the interaction between blockchain and the off-chain com-
ponents. The oracle is connected to a unifying API, which can interact with the
different attributes assessment methods.
There is a decision making module in our generic architecture that decides
whether an item is considered authentic or not: it takes the different assessments
for the available methods and decides whether the claimed physical attributes
match the observed ones well enough. The decision making module can be either
deployed on-chain or off-chain, depending on the specific situation. In the context
where decision is done in a proprietary, confidential way by a single authority,
the decision making module would be kept off-chain. If the decision making has
been established by consensus from a set of authorities, and its workings need
not be kept confidential, the decision making module can be kept on blockchain
to ensure the transparency of its rules and implementation.
2.2 Food Fraud Detection
Food fraud encompasses the deliberate and intentional substitution, addition, or
misrepresentation of food, food ingredients or food packaging for economic gain.
Introducing blockchain into supply chain with Internet of Things (IoT) [6] enable
real-time monitoring of physical movement via tags on products and immutable
food provenance recorded on blockchain. In additional, the smart contract on
blockchain can be used to enable compliance checking. The provenance infor-
mation or the authenticate food can be used to detect simulation, where the
illegitimate product is designed to look like the legitimate product with the
same label, or overrun, where the legitimate product is made in excess of pro-
duction agreements. In these two types of food fraud, there are duplicated labels.
Although it is difficult to determine which one is authentic, and which one is
simulation or overrun based on the digital representation only. Diversion means
the sale or distribution of legitimate products outside of intended markets. If
the condition of sale and distribution associated with the products is recorded
in blockchain, smart contract can help to do automatic compliance checking
against legislation. However, other types of food fraud, like adulteration, where
a component of the finished products is fraudulent, can not be detected by using
provenance information because the coupling between the physical food and
its digital representation is loose when only basic tagging techniques are used.
Adulteration becomes automatically detectable when the binding between the
physical food and its digital representation is tighten by any of the attributes
assessment techniques.
70 S. K. Lo et al.
3 Food Fraud Detection System Design
We instantiate a food fraud detection system design based on the proposed
generic architecture. The system helps to manage the knowledge of differenti-
ating fraud from non-fraud. The knowledge about food fraud is accumulated
through analysis of multimodal data (regular camera, thermal camera, and spe-
cially designed sensors) as well as the interaction between food producers, inde-
pendent food experts, and recipients in a food supply chain. A more detailed
machine learning assessment module of the system is shown in Fig.3. Although
we demonstrate the system design using an image processing technique, the
architecture can use other machine learning techniques.
As discussed above, the existing methods of assessing the integrity of the
physical attributes of food products are either expensive (some require com-
plicated lab procedures) or vulnerable to attack (an adversary can easily swap
the RFID tag for the physical food products). This food fraud detection system
focuses on visible, distinguishable features of food product that can be captured
by a regular camera, thermal camera, or other specially designed sensors. Take
wild salmon as an example. Wild salmon get their characteristic colour from
what they eat. The unique colour reflects the diet of shrimp and krill. Salmon
from different geo-location consumes a different proportion of carotenoid-rich
creatures, which affects how pink or red salmon becomes.
Classier
Machine Learning on Visual Features
Model Database Food Data
Model
Matchmaker
Classicaon
Explainer
Out-of-distribuon
Detector
Classier
Classier
Dataset
Dataset
Dataset
Fig. 3. Machine learning assessment of our food fraud detection system
The main components of this system is a data repository, a knowledge base
that captures the differences between fraud and non-fraud food products and
a blockchain-based trust management module. The knowledge is represented
as machine learning models that make use of food product data collectible by
food suppliers and recipients to classify frauds. The data and models are often
provided by food suppliers or independent parties. A consumer can make use of
these models to classify whether the food product she purchases is fraud.
There are incentives for a high quality food product supplier to provide data
containing identifiable features to differentiate their high quality products. Sim-
ilarly, food fraudsters also have the motivation to control and manipulate the
data/model in the system to confuse the consumers to gain benefit.
Digital-Physical Parity for Food Fraud Detection 71
3.1 Users
The users in the system are:
1. Food product suppliers and experts who are knowledgeable about the differ-
ence between fraudulent and authentic food products.
2. Recipients in a food supply chain who can collect data of a particular food
product.
3. Machine learning developers that provide services for recipients to check food
fraud based on the data provided by recipients.
3.2 Data Provenance
For food products with potential digital features, aggregating various test meth-
ods for transparency and reproducibility is essential. As mentioned in Sect. 2,we
apply the blockchain technique to enable tracking data and model provenance
and linking them to the party for accountability.
All the datasets within the food database and the classifiers within the model
database are registered on the blockchain, thus, publicly available to all the
users of the system. Other than registration, all the activities conducted on the
datasets and the models have separate records on the blockchain. The tamper-
proof log of events on blockchain provides the connection between the data
(as the digital representation of the food product) and the historical activities
executed on the data. Machine learning models are viewed as a type of data.
A simplified data analytics life cycle is depicted in Fig. 4a. Food suppliers and
recipients contribute to this process by uploading batch data or new data item
of a food product. Machine learning developers contribute to this process by
cleaning the raw data and training/re-training models that capture the knowl-
edge from the food experts. The data analysts register all the analytic activities,
for example, who does cleaning on which dataset at what time. Every activity
record on blockchain is signed by the person who conducts the activity using
their digital signature. Such information allows tracing back to data analysis
activities that have been conducted on a particular data. It allows the system
users to track the changes of food fraud detection knowledge represented by
the data and the models and the system to quantify the contribution activities
towards the final model.
The incentive mechanisms discussed in Sect. 3.3 uses the data provenance
and food traceability information recorded on blockchain to distribute reward
to the contributors and honest participants in the supply chain.
3.3 Trust Management Through Incentive
We use blockchain to provide an incentive infrastructure for the food fraud detec-
tion system. With cryptocurrency, blockchain can also provide a trading infras-
tructure that enables contingent payment implemented as a smart contract for
trading items registered on the blockchain. In the context of incentive, honesty
72 S. K. Lo et al.
and trust is the “tradable” item. We introduce two coins as the incentive mecha-
nisms. Contribution-coin is the incentive for ML scientists and experts to use the
system and share their knowledge and data. Honesty-coin is used as the reward
to the honest recipients in the supply chain.
Reward to Contribution. There are two mechanisms to reward contribution.
One is a rule-based automated mechanism that rewards the users of the platform
based on their contribution. The contribution to the platform can be roughly
quantified based on metrics proposed by machine learning experts. The value of
data is determined by its size and diversity. A data has more value if it is different
from any of the data in the system, for example, data from a new food product,
or data from a new feature of a food product. Model with higher accuracy has
more value to the system. How to calculate the contribution based on these
or more metrics is our future work. Point-to-point reward model as shown in
Fig. 4a will be implemented. The reward to a certain model is split between the
developer who cleans the data and the developer who trains the model if more
than one developers are involved in this data analytics process. How to split the
reward is decided by machine learning developers.
The second is a human-driven point-to-point reward mechanism, like the
digital reward system on social media where rewards can be sent between users
arbitrarily. There will be constraints for the reward, as the monetary limit. Since
all the reward transactions are recorded on the blockchain, they are transparent
for public auditing. For example, ML scientist A is rewarded xContribution-
coins from supplier B, a link is established between A and B with weight x.If
the supplier is fraudster or colluding with ML scientist, it is supposed to have
unusual patterns [2].
Reward to Honesty. The food fraud detection is primarily to check compliance
between the authenticity information on blockchain and the food product data
uploaded by recipients. As shown in Fig. 4b, any participants in supply chain can
report a potential fraud case via our system. In the case that the food product
complies with the authenticity information provided by the farm, some honesty-
coins are rewarded equally to all the participants in the supply chain according to
the food traceability information on the blockchain, With this incentive reporting
mechanism, it motivates every participant in the same line of supply chain to
cross check the product that arrived at their position.
However, in the case of negative result, the result of fraud detection does not
indicate which participant in the food supply chain is the fraudster. In the case
that the food product does not comply with the authenticity information, a food
expert is selected and requested to validate the result. If the result is invalid,
a machine learning developer is alerted and requested to re-train the model. If
the food expert confirms the validity of the result, the honesty-coin own by all
the previous participants in the supply chain will be minused off to pay the
reporter. This mechanism punishes all the participants in the supply chain as an
Digital-Physical Parity for Food Fraud Detection 73
Upload Data
Clean Data
Consumer / Supplier
Train Model
Add More Data
Re-train
Contributors
ML Scienst
Pre-process
Data
Food Expert
Rule-based
Honesty Reward
Point-to-point
Reward
(a) Simplified data analytics life cycle and
contribution coin distribution
Farm
Abaoir
Retailer
Consumer
Meat Market
Honesty Reward
Fraud Detecon
(b) Simplified beef supply chain
and honesty-coin distribution
Fig. 4. Coins distribution
incentive for honest participants to find out the food fraudster or partner with
other participants with a higher balance of honesty coin.
3.4 Off-Chain Components
As the blockchain does not have the scalability to execute computation heavy
process on-chain nor it could store big size of raw data on-chain, we have chosen
to execute our machine learning model and store all the dataset for the model off-
chain. The inherent disadvantage of using off-chain is that there are no native way
for blockchain to fetch data from off-chain. Hence, the interoperability between
the off-chain components and blockchain can be achieved with the use of an
oracle. Oracle is needed to inject data from off-chain into blockchain. Details are
discussed in Sect. 4. The training dataset and the model are all stored off chain.
Off-chain databases and modules are being described below:
Food database—A collection of datasets that show the difference between
fraudulent and non-fraudulent of various food products. Food fraud is a huge
market, and a continuous data collection mechanism is vital to help detection
techniques keeping up with fraud techniques with new data collection devices
and knowledge built on top of new datasets;
Model database —Storing various classifiers and their associated metadata. We
assume that the classifiers are learned of the system, based on the datasets
within the food database. Every classifier is aimed to distinguish fraudulent
food from non-fraudulent food under certain circumstances. The definition of
fraud is context specific. One example is to distinguish grain-fed beef from
grass-fed beef, as discussed in Sect. 5. All the newly uploaded product data
and the corresponding detection results are stored, and a machine learning
developer is selected regularly to double check and confirm the result. Such
dataset with a label will be used to improve the performance of the models.
74 S. K. Lo et al.
Model matchmaker—When a recipient submits a query with the data she
collects about a specific food product to the system, the system needs to
search for its model database for suitable models to answer the query. The
“model matchmaker” is responsible for this task.
Out-of-distribution detector—A user-submitted query is likely to contain pat-
terns that has never been seen by a model. It is important to detect such a
mismatch to avoid arbitrary classifications. The “Out-of-distribution detec-
tor” is responsible for checking whether the user-submitted data has different
distribution with the data on which a model is trained. It may trigger the
re-training of a model based on changing data or training of a new model
when data distribution has a significant change.
4 Implementation
An overall deployment architecture of the food fraud detection system is shown
in Fig. 5. It has four main components: (1) a blockchain with registries and incen-
tives implemented in smart contracts; (2) classifiers and an out-of-distribution
detector which are hosted on AWS; (3) an Amazon S3 bucket for storing images
uploaded from the users and other datasets; and (4) an Oraclize-based oracle as
the main connector between the blockchain and AWS.
Incenve coin contract
Blockchain
Dataset Registry
Registries
Off-chain
Food supply
chain registry
Food Product
Registry
AWS Cl oud
Model
Out-of-distribuon
Detector
Model registry
1
Classifier
Labeled dataset
Uploaded images
Out of distribuon
dataset
Classificaon
result
2
3
4
I
Fig. 5. Overall deployment diagram
4.1 Registries as Smart Contracts
We implemented the food fraud detection system with Ethereum Rinkeby Test-
net. All the datasets within the food database and the classifiers within the
model database are registered to smart contracts called dataset registry and
model registry. The registries store the metadata of the datasets (description,
ownership, the location of the data) and the classifiers (description, ownership,
purpose, accuracy). The raw data (the machine learning models and photos of
food products) are stored off-chain in the Amazon S3 bucket. A registry, food
supply chain registry, is used to establish the relationship of all the participants
Digital-Physical Parity for Food Fraud Detection 75
in the supply chain for a food product. The stakeholders of a supply chain play
an essential role in ensuring a customer gets a genuine product as described on
the its label by the food supplier. Hence, all the details of related participants for
food products will be recorded on the blockchain. Another registry, food product
registry, is used to register the metadata of the food product image submitted
by the recipient. Metadata includes the checksum of the image and a pointer
that links to the raw image. Once a food product image is labeled and used to
train a model, it is removed from the food product registry and is registered in
the dataset registry as part of a dataset.
4.2 Incentive Mechanisms as Smart Contracts
We introduced contribution-coin and honesty-coin to incentivize the honesty of
food supplier and contribution of participants towards the system. Both coins
are implemented in smart contracts that are compliant to the ERC20 token
standard2. All the balance of token owners will be recorded by the token contract.
We have a ratioConfig(address tokenOwner, uint ratio) function to allow
authorized admins (e.g., verified ML scientists) to adjust the ratio pertaining to
the distribution of the tokens. For contribution-coin, the experts that verify the
accuracy of the newer model will be permitted to set the coin distribution ratio
according to the task description entered by the ML scientist. Some of the tasks
related to retraining a model include preprocessing and cleaning dataset.
Contribution-coin is rewarded to the ML scientists for their contributions to
improve and maintain the machine learning model. Experts are also rewarded
for checking the result of the classifier and label given on the training datasets
periodically. The initial distribution ratio of coin is set to 6:4 between the ML
scientist and experts that verify the classifier and label. Honesty-coin is given
to participants of the supply chain for their honesty in supplying genuine food
products to the public. If the classifier determined that a food product is genuine
as stated on its food label, every participant in the supply chain for that food
product will be rewarded Honesty-coin based on the weight predefined in the
food supply chain registry
4.3 Off-Chain Storage and Model Execution
The images of food products uploaded by recipients are stored in off-chain stor-
age. In our prototype, we have opted for AWS S3 due to its availability and
resilience of the stored data. A pointer that links to the image stored in AWS
S3 and checksum of the image will be recorded on the blockchain. Storing only
the metadata of images on the blockchain eliminates the high cost of storing
images on and the storage limitation of blockchain while allowing detection of
any tampering of the images. The pointer and the checksum of the images will
be stored on-chain via a transaction.
2https://theethereum.wiki/w/index.php/ERC20 Token Standard.
76 S. K. Lo et al.
The trained models and the training datasets are also stored off-chain. The
execution environment of the ML model is running on an Ubuntu 16.04 LTS AWS
EC2 instance. The machine learning model is written in the Python language
with TensorFlow. This module provides two main operations: (1) image data
upload; and (2) image data classification. The latter returns classification results
together with saliency maps of the result as well as the confidence that the input
image is within the knowledge scope of this model used.
4.4 Oracle for On-Chain and Off-Chain Interoperation
An oracle is a mechanism that fetches data from the external world to the isolated
execution environment of a blockchain. We have selected Oraclize3for our oracle
implementation. It provides various proof mechanisms to ensure the validity of
the information acquired from the data source. In our prototype, Oraclize is used
to obtain the result of the ML classifier for food fraud detection and distribution
of the reward coins to the participants. It is triggered once the result is generated
from the model classifier. The oracle will inject the result of the classification
into the blockchain and use it as an input for the incentive coin distribution.
5 Example Case Study
We use a simple case study of a beef supply chain to demonstrate the feasibility
of our food fraud detection system, which is used to distinguish grain-fed beef
(lower quality and price) from grass-fed beef (higher quality and price).
Using the food fraud detection system, a beef recipient can verify the claimed
attributes of the beef product (written on the label of in the digital represen-
tation). To use the system, the recipient needs to take photo of a full top view
of the beef product under good lighting, currently without plastic packaging or
other covers, and upload the image to our system. Once the image has been
uploaded successfully, the checksum of the image will be calculated, and a script
will invoke the Food Product Registry smart contract to add a data item. The
pointer to the image and the checksum of the image will be entered as input
data on the smart contract to record them on the blockchain. The off-chain ML
system is triggered once the data has been recorded on the blockchain. It will
retrieve the input data from the blockchain (pointer and checksum) to down-
load the picture, verify the checksum and run the image through the classifier
to check whether it is a grain-fed beef or grass-fed beef. The result is checked
against the claim, and a threshold-based decision is made and recorded in the
result database, triggering the incentive mechanisms.
5.1 Machine Learning for Classifying Beef Types
Grain-fed beef and grass-fed beef are visually different as grain-fed beef has more
fat. However, the difference can be subtle and overwhelm consumers. Examples
3http://www.oraclize.it/.
Digital-Physical Parity for Food Fraud Detection 77
of the two types of beef are shown in Fig. 6. We trained a classifier based on a
small set of grass-fed and grain-fed beef images to help detect the fraud.
(a) grain-fed beef (b) grass-fed beef (c) Anomaly
example
(d) Anomaly
example with
known pattern
Fig. 6. Beef images
We built a neural network-based classifier to distinguish grain-fed beef and
grass-fed beef based on the meat texture in input images. The binary classifier
contains two convolutional layers with 24 and 32 filters at the first and the second
layer respectively, and with a kernel size of dimension (5,5), two max pooling lay-
ers, one drop-out layer and two fully connected layers with 64 units and 2 unites
respectively. The texture difference can be learned using a small amount of train-
ing data obtained from supplier and supermarket websites. The model accuracy
is 92.5% on a dataset randomly picked from the Web. The amount of training
data grows as recipients or producers keep collecting and contributing data to
the platform, which enables the classifier to be tuned to distinguish a greater
variety of beef of the two types. The data diversity enhances the capability of
the classifier, thereby enriching the knowledge about food product differences
accumulated in the platform.
The learned “knowledge” of a model is useful when a recipient-collected image
preserves the pattern known by the classifier even though the machine learning
model may not be directly trained from the same type of images. When a beef
steak is cooked, the features differentiating grass-fed from grain-fed are not vis-
ible, as shown in Fig. 6c. However, when a recipient uses a thermal camera to
take a picture of the beef and submits the image, as shown in Fig. 6dtothe
system to check, the pattern of grain-fed beef becomes highly distinguishable as
the fat part has a higher temperature, and thus, a specific model can classify
it correctly. This demonstrates the knowledge capturing capability of machine
learning models in food fraud detection scenario.
5.2 Incentive Mechanism
In the proposed incentive mechanism, the two types of coins are the Honesty-
coin and Contribution-coin, which are distributed as follows. Honesty-coin will
be distributed to all the participants recorded on the supplier registry via their
accounts if the product bought by the final recipient is verified to be genuine.
Once the result from the classifying process is available, the system triggers the
incentive coin smart contract. If the result shows “genuine”, the smart contract
78 S. K. Lo et al.
will distribute Honesty-coin to participants according to a predefined split. In
the implementation and the case study, we have set equal split for every partic-
ipant. If the result indicates fraud, the involved participants could be penalized;
however, in a decentralized system, that would require the participants to put
up a stake upfront from which penalties could be deducted.
The system tracks out-of-distribution cases. Once the occurrence reaches a
certain threshold, the ML scientists will be notified to check and tune their
model or create a new model. The new model will be adjusted by checking all
the recipient-submitted image input. When a model is updated, a smart contract
with oracle will be used to inject the model’s performance in term of accuracy
to the blockchain. The actual model will be stored in cloud data registry. If the
updated model achieves higher accuracy than the previous one, contribution-
coin will be distributed to the scientist, data contributors and experts who help
to verify the performance of the updated model. Currently, the coins are split
among all parties in a pre-defined fix ratio.
5.3 Discussion
Our system provides a transparent platform for recipients to verify claims about
a food product against its physical attributes. It ensures the originality and
integrity of uploaded data and allows tracing back to all participants in case of
food fraud. By using incentives, the system encourages participants continuous
engagement.
Our design takes future needs into consideration. The classifiers are loosely
coupled with other components within the system, thus allowing for ML models
to be swapped and modified according to the needs and data. Furthermore,
the architecture supports other methods for assessing physical attributes of a
product, like isotopic or genomic analysis.
There are limitations arise from using blockchain and machine learning for
food parity. There are chances where the beef classifier itself might be rigged for
maximum economy benefit for all the participants in the supply chain except
the buyer. Although we added the expert role as a trusted intermediary to verify
and ensure the trustworthiness of the dataset and classifier, but influential sup-
ply chain market player might still be able to find way to manipulate the result
or machine learning model outside of blockchain and supply chain. As partipants
on blockchain are transparent, supply chain participants might be able to com-
municate wth each other outside of the supply chain to gain mutual consensus to
work together to commit fraud. Another problem would be malicious recipients
submitting fake beef product. For example, they could have gotten good quality
product, but ended up substituting beef from others as input to the system for
personal gain or to purposely jeopardize the partipants in the supply chain.
6 Conclusion and Future Work
We proposed a generic architecture that connects physical food products in a
supply chain with their digital representation. The architecture uses blockchain
Digital-Physical Parity for Food Fraud Detection 79
for immutability and an incentive system. We instantiate this architecture with a
concrete system using machine learning on visual features of beef, to check if the
digital claims on the beef product match the physical attributes. By evaluating
our system with a real-world scenario on distinguishing the types of beef, we
determine that it is feasible to implement the components in our proposed generic
architecture to achieve parity between a physical food product and its digital
representation. To the best of our knowledge, this is the first research work that
combines physical attribute of food products with their digital representation
via blockchain and machine learning. Our future work will focus on rewarding
participants relative to their contribution, and on introducing a reputation coin
that can be used to assess the trustworthiness of participants.
Acknowledgements. This research is supported by the Science and Industry Endow-
ment Fund of Australia.
References
1. Food fraud vulnerability assessment and mitigation-are you doing enough to pre-
vent food fraud? Technical report, PWC (2016)
2. Fleder, M., Kester, M.S., Pillai, S.: Bitcoin transaction graph analysis. arXiv
preprint arXiv:1502.01657 (2015)
3. Lo, S.K., Xu, X., Chiam, Y.K., Lu, Q.: Evaluating suitability of applying
blockchain. In: The 22nd ICECCS, November 2017
4. Osorio, M.T., Moloney, A.P., Schmidt, O., Monahan, F.J.: Beef authentication
and retrospective dietary verification using stable isotope ratio analysis of bovine
muscle and tail hair. J. Agric. Food Chem. 59(7), 3295–3305 (2011)
5. Staples, M.: Risks and opportunities for systems using blockchain and smart con-
tracts. Technical report, Data61 (CSIRO), Sydney (2017)
6. Tian, F.: A supply chain traceability system for food safety based on HACCP,
blockchain & internet of things. In: The 14th International Conference on Service
Systems and Service Management, June 2017
7. Uludag, U., Pankanti, S., Prabhakar, S., Jain, A.K.: Biometric cryptosystems:
issues and challenges. Proc. IEEE 92(6), 948–960 (2004)
8. Xu, X., Lu, Q., Liu, Y., Yao, H., Zhu, L., Vasilakos, T.: Designing blockchain-
based applications: a case study for imported product traceability. Futur. Gener.
Comput. Syst. 92, 399–406 (2019)
9. Xu, X., et al.: The blockchain as a software connector. In: The 13th Working
IEEE/IFIP Conference on Software Architecture, April 2016
10. Zhou, X., Taylor, M.P., Davies, P.J., Prasad, S.: Identifying sources of environmen-
tal contamination in european honey bees (Apis mellifera) using trace elements and
lead isotopic compositions. Environ. Sci. Technol. 52(3), 991–1001 (2018)
... In particular, BC technology provides no protection against data that are intentionally manipulated prior to their validation on the network (Schmidt and Wagner, 2019). Moreover, reading information from the BC is an issue because the information accessed by consumers on a certain product may be stored offchain or may not actually refer to that specific product, given that commonly used smart labels can easily be cloned and then re-applied to counterfeited products (Lo et al., 2019;Kumar et al., 2020). ...
... QR codes vs NFC tags or manual vs automatic data entry), previous studies (e.g. Galvez et al., 2018;Alzahrani and Bulusu, 2019;Azzi et al., 2019;Lo et al., 2019) implicitly suggest that the different feeding/reading options used to complement the core BC technological properties can assure protection against counterfeiting to different extents, but again the view remains fragmented. ...
... QR codes and barcodes) as well as high-tech labelling (e.g. NFC and RFID tags) that allow users to easily access product-related information via their smartphones (Creydt and Fischer, 2019) (1) Tags cloning: smart labels commonly used by these systems can be subject to cloning and reuse on counterfeit products (Lo et al., 2019) (2) Manipulation of product-related information: these systems are vulnerable to information manipulation after data entry because data is stored on centralized servers that allow modifications (Biswas et al., 2017) (3) Poor quality of the input data: track-and-trace systems can be fed with information that does not reflect reality if no specific activities and controls are carried out to verify data-entry quality Note(s): *With the exception of those seals that need regulatory approval and can be issued only after on-field audits (e.g. ...
Article
Purpose The purpose of this paper is to contribute to the debate on blockchain (BC) adoption for preventing counterfeiting by investigating BC systems where different options for BC feeding and reading complement the use of BC technology. By grounding on the situational crime prevention, this study analyses how BC systems can be designed to effectively prevent counterfeiting. Design/methodology/approach This is a multiple-case study of five Italian wine companies using BC to prevent counterfeiting. Findings This study finds that the desired level of upstream/downstream counterfeiting protection that a brand owner intends to guarantee to customers through BC is the key driver to consider in the design of BC systems. The study identifies which variables are relevant to the design of feeding and reading processes and explains how such variables can be modulated in accordance with the desired level of counterfeiting protection. Research limitations/implications The cases investigated are Italian companies within the wine sector, and the BC projects analysed are in the pilot phase. Practical implications The study provides practical suggestions to address the design of BC systems by identifying a set of key variables and explaining how to properly modulate them to face upstream/downstream counterfeiting. Originality/value This research applies a new perspective based on the situational crime prevention approach in studying how companies can design BC systems to effectively prevent counterfeiting. It explains how feeding and reading process options can be configured in BC systems to assure different degrees of counterfeiting protection.
... Therefore, in order to reach consumers during every interaction with food (such as buying it at a shop, eating it at home or in a restaurant and so on), it is necessary to think about methodological and technological solutions that are able to overcome the current concept of a physical label (Lo et al., 2019). Indeed, the physical space on the label is not infinite and a selection of the printed information is needed. ...
Article
Today, the world faces a big challenge about food quality and safety. Consumers are becoming more careful during their food choices, focusing on food information. Drawing on the Industry 4.0 technologies and methodologies, food companies could adopt traceability systems that are able to collect various types of information along the food supply chain, to satisfy regulation but also to make their work transparent. With the aim of overcoming the current shortcomings in the food traceability framework (e.g., the absence of data visualisation solutions or tools for supply chain actors and consumers, adopting standards for external traceability and data identification, and fulfilling the four Industry 4.0 development dimensions), and considering the sustainability challenges, the aim of this study is to suggest a Framework for Voluntary Food Traceability based on digital technologies and explore its application by means of a case study in an organic olive oil company. The Framework, composed of three building blocks, allows the company to: map supply chain processes, identifying tasks and processes where data was generated, its format and the supply chain owner; select and use technologies to collect and analyse traceability data; and communicate to the end consumer comprehensible and useful food information.
... • AI-based Digital-Physical Parity [20] is a solution that uses machine learning models to distinguish between grass-fed and grain-fed beef. Blockchain is adopted to store the respective dataset registry and model registry. ...
Preprint
Full-text available
As the killer application of blockchain technology, blockchain-based payments have attracted extensive attention ranging from hobbyists to corporates to regulatory bodies. Blockchain facilitates fast, secure, and cross-border payments without the need for intermediaries such as banks. Because the blockchain technology is still emerging, systematically organised knowledge providing a holistic and comprehensive view on designing payment applications that use blockchain is yet to be established. If such knowledge could be established in the form of a set of blockchain-specific design patterns, architects could use those patterns in designing a payment application that leverages blockchain. Therefore, in this paper, we first identify a token's lifecycle and then present 15 design patterns that cover critical aspects in enabling the state transitions of a token in blockchain-based payment applications. The lifecycle and the annotated design patterns provide a payment-focused systematic view of system interactions and a guide to effective use of the design patterns.
Article
Increase in global population growth and the consequent rise in the demand for food while ensuring quality, preventing wastage, avoiding deforestation, and carbon footprinting has put tremendous pressure on the global food supply chain (FSC). Due to fast computing facilities and availability of data, blockchain technology has emerged as a potential system that may help in developing a safe and secure FSC. Blockchain has gained significant attention among researchers worldwide, which is evident from the extensive literature generated on this subject within a span of a few years. Through this paper, the authors attempt to showcase the immense scope and significance of blockchain in FSCs by examining and analyzing the existing studies. In addition to providing an overview of the application of blockchain in FSCs, this study also discusses challenges such as scalability, interoperability, and high cost and suggests potential solutions for the existing difficulties in adopting blockchain in FSCs. Additionally, a bibliometric analysis is provided to acquaint researchers and practitioners with the structure and research trends in this area. The study found that researchers have mostly focused on the applicability of blockchain for product authentication, finance, and logistics. Further, blockchain is likely to become a dominant technology for enhancing transparency and traceability, reducing risk and, most importantly, enhancing trust among different stakeholders in the area of FSC.
Chapter
Situation faced: One large set of problems in Australian agricultural supply chains comprises liquidity and payment certainty: Farmers are typically paid for their goods after long delays, and sometimes payment fails altogether. At the same time, first buyers of crops (exporters, traders, processors, feedlots) have limited funds as they, like the farmers, have little access to financing, such as obtaining a loan against a quantity of the harvested crops. Financiers are often unwilling or unable to offer financing against such assets because of insufficient visibility—that is, the ability to see and verify the assets’ information, or the high cost of obtaining it. These liquidity and financing shortages are harmful to the Australian agricultural industry.
Article
Unlike previous reviews, most of which focus on a field and use qualitative methods to analyze blockchain technology, herein main path analysis was used to determine the key route of citations and identify the developmental trajectories of blockchain technology and its major themes. A total of 3159 blockchain-related articles published through August 5, 2019, were gathered from the Web of Science. In all, 2478 articles formed the citation network, 27 of which were on the key route. These 27 key papers on blockchain represented the four developmental stages. Stage 1 focused on challenges to bitcoin. Stage 2 focused on smart contract issues. Stage 3 focused on opportunities, challenges, and development in blockchain. Finally, Stage 4 focused on smart contract applications, especially in e-healthcare, where blockchain technology is paving the way to a quicker, more secure, and more efficient means of storing and accessing medical data. The Girvan-Newman clustering technique was used to classify articles, and keyword analysis was used to analyze the top six major themes. These were bitcoin security, bitcoin financial applications, e-healthcare, Ethereum smart contracts, security and privacy in IoT, and energy. Main path analysis was also used to analyze their developmental trends. The results have significant value for both academics and practitioners.
Article
Full-text available
Trace element concentrations (As, Mn, Pb and Zn) and Pb isotopic compositions were analysed in honey bees, wax and honey along with co-located soil and dust samples from Sydney metropolitan and Broken Hill, Australia. Compared with the other trace elements, Pearson correlations show that Pb concentrations in soil and dust had the strongest relationship to corresponding values in honey bees and their products. Dust Pb was not only highly correlated to corresponding soil values (r = 0.806, p = 0.005), it was the strongest predictor of Pb concentrations in honey bees, wax and honey (p = 0.001, 0.007, 0.017, respectively). Lead isotopic compositions (²⁰⁶Pb/²⁰⁷Pb and ²⁰⁸Pb/²⁰⁷Pb) showed that honey bees and their products from Broken Hill were nearly identical (95–98 %) to the composition of the local ore body. Samples of honey bees and their products collected from background sites adjacent to national parks in Sydney had Pb isotopic compositions (²⁰⁶Pb/²⁰⁷Pb = 1.138–1.159, ²⁰⁸Pb/²⁰⁷Pb = 2.417–2.435) corresponding to local geogenic values (²⁰⁶Pb/²⁰⁷Pb = 1.123–1.176, ²⁰⁸Pb/²⁰⁷Pb = 2.413–2.500). By contrast, honey bees and their products from Sydney metropolitan (²⁰⁶Pb/²⁰⁷Pb = 1.081–1.126, ²⁰⁸Pb/²⁰⁷Pb = 2.352–2.408) were similar to aerosols measured during the period of leaded petrol use (²⁰⁶Pb/²⁰⁷Pb = 1.067–1.148, ²⁰⁸Pb/²⁰⁷Pb = 2.341–2.410). These measurements show Pb concentrations and its isotopic compositions of honey bees and their products can be used to trace both legacy and contemporary environmental contamination, particularly where sources are well documented. Moreover, this study demonstrates that legacy Pb emissions continue to be remobilised in dust, contaminating both food and ecological systems.
Technical Report
Full-text available
Blockchain technologies originally emerged to support new forms of digital currency, but now hold promise as a new foundation for transactions in society. A blockchain is both a database recording transactions between parties, and also a computational platform to execute small programs (called ‘smart contracts’) as transactions. A blockchain is a distributed database, replicated across many locations and operated jointly by a collective. Blockchains transactions can support services for payments, escrow, notarisation, voting, registration, and process coordination. These are key in the operation of government and industry. Conventionally, these services are provided by speci c trusted third-parties such as banks, legal rms, accountancy rms, government agencies, and service providers in speci c industries. With a blockchain-based system, rather than relying on third-party organisations, we could instead choose to rely on the blockchain software and on a majority of the collective that jointly operates the blockchain system. The report describes some of the technical risks and opportunities in the application of blockchain technologies within government and industry, and how to assess whether blockchain-based systems will meet critical requirements. The project explores this primarily through the description and analysis of high-level design alternatives for illustrative ‘use cases’. Three use cases have been selected after a number of initial workshops and preliminary research: remittance payments, open data registries, and agricultural supply chain. These provide reasonable coverage of various kinds of requirements and regulatory concerns, against which we can evaluate design alternatives, and in turn learn more general lessons about blockchain technologies. In addition to this design-based analysis, we also report on some empirical results from testing prototype implementations. Compared to conventional centralised databases and computational platforms (on-premises or cloud), blockchains can reduce some counter-party and operational risks by providing neutral ground between organisations. Blockchain technologies may provide advantages for integrity and non-repudiation. However, they also currently have limitations for con dentiality, privacy, and scalability. For latency and availability, reading is improved but writing is worsened. Blockchains are also subject to a di erent cost model. Digital currency transfer and long-term storage of transactional data may be less expensive. However, program execution and storage of big data may be more expensive. Public blockchains provide very low barriers to entry for new participants, which can facilitate competition, innovation, and productivity. However, they do not mandate authentication of those participants, which creates challenges for regulation of money laundering, terrorism nancing, and tax avoidance. Private blockchains can impose more controls on authentication and access, which can partly address those regulatory concerns. Still, for competitors within an industry consortium, private blockchains may not be private enough to provide normal levels of commercial con dentiality for business operations, competitive position, and customer relationships. When assessing business risk, regulatory acceptance, and assurance arguments for a blockchain-based system, we need to consider not just the blockchain, but also all of the other components that are integrated in the design of the whole system. Other components will provide user interfaces, cryptographic key management, and o -chain databases, communications, and processing. Judicious use of these other components may mitigate blockchain’s risks while still leveraging blockchain’s opportunities. Finally, blockchains are still a rapidly evolving technology, with ongoing developments especially to improve scalability and con dentiality. Globally, governments, enterprises, and startups are exploring the technology/ market t in a wide variety of use cases and for a wide variety of requirements and regulatory demands. There is still much that is unknown about the development of trustworthy blockchain-based systems. Further research is required to improve our knowledge about how to create blockchain-based systems that work, and how to create evidence that blockchain-based systems will work as required.
Article
Full-text available
In traditional cryptosystems, user authentication is based on possession of secret keys; the method falls apart if the keys are not kept secret (i.e., shared with non-legitimate users). Further, keys can be forgotten, lost, or stolen and, thus, cannot provide non-repudiation. Current authentication systems based on physiological and behavioral characteristics of persons (known as biometrics), such as fingerprints, inherently provide solutions to many of these problems and may replace the authentication component of traditional cryptosystems. We present various methods that monolithically bind a cryptographic key with the biometric template of a user stored in the database in such a way that the key cannot be revealed without a successful biometric authentication. We assess the performance of one of these biometric key binding/generation algorithms using the fingerprint biometric. We illustrate the challenges involved in biometric key generation primarily due to drastic acquisition variations in the representation of a biometric identifier and the imperfect nature of biometric feature extraction and matching algorithms. We elaborate on the suitability of these algorithms for digital rights management systems.
Article
Blockchain technology enables decentralization as new forms of distributed software architectures, where components can reach agreements on the shared system states without trusting on a central integration point. Since blockchain is an emerging technology which is still at an early stage of development, there is limited experience on applying blockchain to real-world software applications. We applied blockchain application design approaches proposed in software architecture community in a real-world project called originChain, which is a blockchain-based traceability system that restructures the current system by replacing the central database with blockchain. In this paper, we share our experience of building originChain. By using blockchain and designing towards security, originChain provides transparent tamper-proof traceability data with high availability and enables automated regulatory-compliance checking and adaptation in product traceability scenarios. We also demonstrate both qualitative and quantitative analysis of the software architecture of originChain. Based on our experience and analysis, we found that the structural design of smart contracts has large impact on the quality of the system.
Article
Stable isotope ratio analysis (SIRA) was used as an analytical tool to verify the preslaughter diet of beef cattle. Muscle and tail hair samples were collected from animals fed either pasture (P), a barley-based concentrate (C), silage followed by pasture (SiP), or silage followed by pasture with concentrate (SiPC) for 1 year (n = 25 animals per treatment). The (13)C/(12)C, (15)N/(14)N, (2)H/(1)H, and (34)S/(32)S isotope ratios in muscle clearly reflected those of the diets consumed by the animals. By applying a stepwise canonical discriminant analysis, a good discrimination of bovine meat according to dietary regimen was obtained. On the basis of the classification success rate, the (13)C/(12)C and (34)S/(32)S ratios in muscle were the best indicators for authentication of beef from animals consuming the different diets. Analysis of (13)C/(12)C and (15)N/(14)N in tail hair sections provided an archival record of changes to the diet of the cattle for periods of over 1 year preslaughter.
  • M Fleder
  • M S Kester
  • S Pillai
Fleder, M., Kester, M.S., Pillai, S.: Bitcoin transaction graph analysis. arXiv preprint arXiv:1502.01657 (2015)