ArticlePDF Available

Abstract and Figures

Prior work has shown Convolutional Neural Networks (CNNs) trained on surrogate Computer Aided Design (CAD) models are able to detect and classify real-world artefacts from photographs. The applications of which support twinning of digital and physical assets in design, including rapid extraction of part geometry from model repositories, information search & retrieval and identifying components in the field for maintenance, repair, and recording. The performance of CNNs in classification tasks have been shown dependent on training data set size and number of classes. Where prior works have used relatively small surrogate model data sets (< 100 models), the question remains as to the ability of a CNN to differentiate between models in increasingly large model repositories. This paper presents a method for generating synthetic image data sets from online CAD model repositories, and further investigates the capacity of an off-the-shelf CNN architecture trained on synthetic data to classify models as class size increases. 1,000 CAD models were curated and processed to generate large scale surrogate data sets, featuring model coverage at steps of 10◦, 30◦, 60◦, and 120◦ degrees. The findings demonstrate the capability of computer vision algorithms to classify artefacts in model repositories of up to 200, beyond this point the CNN’s performance is observed to deteriorate significantly, limiting its present ability for automated twinning of physical to digital artefacts. Although, a match is more often found in the top-5 results showing potential for information search and retrieval on large repositories of surrogate models.
Content may be subject to copyright.
Available online at www.sciencedirect.com
Procedia CIRP 00 (2019) 000–000 www.elsevier.com/locate/procedia
31st CIRP Design Conference 2021 (CIRP Design 2021)
Distinguishing artefacts: evaluating the saturation point of convolutional
neural networks
Ric Reala,, James Gopsilla,b, David Jonesa, Chris Snidera, Ben Hicksa
aDesign Manufacturing Futures Lab, University of Bristol, UK
bCentre for Modelling and Simulation, Bristol, UK
* Corresponding author. E-mail address: ric.real@bristol.ac.uk
Abstract
Prior work has shown Convolutional Neural Networks (CNNs) trained on surrogate Computer Aided Design (CAD) models are able to detect
and classify real-world artefacts from photographs. The applications of which support twinning of digital and physical assets in design, including
rapid extraction of part geometry from model repositories, information search & retrieval and identifying components in the field for maintenance,
repair, and recording. The performance of CNNs in classification tasks have been shown dependent on training data set size and number of
classes. Where prior works have used relatively small surrogate model data sets (<100 models), the question remains as to the ability of a CNN
to dierentiate between models in increasingly large model repositories.
This paper presents a method for generating synthetic image data sets from online CAD model repositories, and further investigates the capacity
of an o-the-shelf CNN architecture trained on synthetic data to classify models as class size increases. 1,000 CAD models were curated and
processed to generate large scale surrogate data sets, featuring model coverage at steps of 10, 30, 60, and 120degrees.
The findings demonstrate the capability of computer vision algorithms to classify artefacts in model repositories of up to 200, beyond this point
the CNN’s performance is observed to deteriorate significantly, limiting its present ability for automated twinning of physical to digital artefacts.
Although, a match is more often found in the top-5 results showing potential for information search and retrieval on large repositories of surrogate
models.
©2021 The Authors. Published by Elsevier Ltd.
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review under responsibility of the scientific committee of the 31st CIRP Design Conference 2021.
Keywords: Design Repositories; Search & Retrieval; Convolutional Neural Networks; CNNs; Machine Learning; ML; Synthetic Data; Surrogate Models
1. Introduction
Recent trends in digital design, such as Twinning [6] have
underlined the value of rapid, or fully integrated, synchronisa-
tion between physical and digital domains to accelerate pro-
cesses and enhance analytic capability.
The prototyping process comprises vital iteration between a
multitude of physical and digital models [5,14,16], in which
both physical and digital states must be captured, aligned, and
replicated across domains (i.e. updating of CAD models).
A key technical challenge thereby lies in creating a relation-
ship between physical and digital states, such that each may be
recognised as a counterpart of the other, whilst maintaining this
alignment across multiple versions of a prototype. By automat-
ing the detection of physical models and searching/aligning
against a digital counterpart, scope exists to reduce process time
and cost via physical/digital transition.
Detection has previously utilised physical tagging (e.g. QR
and barcodes) or direct scanning (e.g. photogrammetry), where
these methods typically require modification to the physical
prototype or generation of new digital models, recent works
have demonstrated the ability of Convolutional Neural Net-
works (CNNs) to extract and learn distinguishable features of
an artefact for classification. Thus, enabling rapid association
with existing data, such as retrieving CAD models by taking a
photograph of their real-world counterpart[4].
However, the performance of a CNN is dependent on quan-
tity and quality of training data [1], and the number of classes
between which it must distinguish [2]. Where photographic
training data is often sparse in prototyping, implementing a
CNN becomes a significant challenge [11], thus the real-world
viability of using CNNs for design twinning is not known.
This paper investigates (a) the performance and implemen-
tation challenges associated with CNN use at increasing scale,
and (b) the viability of ’o-the-shelf’ CNN architectures for
2212-8271 ©2021 The Authors. Published by Elsevier Ltd.
This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review under responsibility of the scientific committee of the 31st CIRP Design Conference 2021.
R. Real et al. /Procedia CIRP 00 (2019) 000–000 2
artefact classification. We consider an artefact to be a designed
object, whose form can be distinguished and classified.
The paper proceeds to present related works in the field of
CNN use in design (Section 2), followed by a methodology for
testing CNN scalability (Section 3). Results are then reported
(Section 5) and a discussion ensues with respect to CNN’s and
their ability to support twinning activities in design (Section 6).
The paper concludes by detailing the key findings from the
study.
2. Related work
CNNs in design: The application of CNNs to design is a
rapidly emerging field of inquiry with many potential impacts
across Engineering Design. For example, [17] trained a CNN
on multi-view renders of 3D geometry and used the resulting
CNN to classify other 3D geometry. A potential use case for
this is the matching of similar parts across product families
with a view to reduce the part variety in an organisations sup-
ply chain. [10] have sought to augment depth mapped images
with Neural Networks to develop voxel-based approximations
of objects within a scene, with potential application to design
via providing a means to describe the locations in which prod-
ucts are used and deployed. [4] is seeking to democratise design
by using a CNN as an information search and retrieval tool for
large model repositories, such as Thingiverse and MyMiniFac-
tory. The CNN enables users to simply take a photo of the item
that they wish to print and return the closest matching result in
the repositories dataset. [3] recently demonstrated CNN’s being
able to emulate mathematical and user perceptions on shape
and form. These could be used to check for conformation to
brand identity as well as potential infringements on others. It
could also twin user feedback into the design process where the
CNN will act as market feedback providing real-time assess-
ment of designs as a designer is working on their product’s de-
sign. While such examples demonstrate the exploration of value
of CNNs in design, there remain questions around their perfor-
mance, and challenges in their implementation.
Surrogate models for dataset generation: The challenge
of acquiring datasets, with thousands of images required per
artefact, has previously been a limiting factor in exploring the
utility of CNNs in large-scale classification tasks. With the con-
text of design prototyping requiring sucient data to distin-
guish between 100s of prototype or component versions, this
presents a systemic obstacle of particular importance.
Recent work has shown the viability of synthetic image
dataset generation, whereby a surrogate CAD model is pro-
cessed and rendered with computer graphics software to gen-
erate two-dimensional artefact representations [13,4]. Image
composition, for example lighting, surfaces, background, and
appearance can additionally be tuned to replicate real-world en-
vironmental features [18,11,12].
This method presents an opportunity to leverage existing vir-
tual models for large-scale classification, whereby a CNN is
trained on synthetic images from a surrogate model data set;
thus mitigating the need for ‘real-world’ photographs, and prior
restrictions in data acquisition [4]. To-date however, the eect
of model repository size on the ecacy of this approach has
not been investigated. Where CNN detection accuracy will typ-
ically decrease as repository size increases, the extensive train-
ing sets producible via surrogate models give scope to substan-
tially increase performance.
3. Methodology
The study followed a 5 step computational process to deter-
mine the scaling behaviour of a CNN, for the purposes of twin-
ning physical artefacts with their digital counterpart in large
repositories.
1. Dataset curation
2. Surrogate model curation
3. CNN selection
4. CNN preparation
5. Evaluation
3.1. Dataset curation
The data set used consisted of 1,049 STL files collected from
the model sharing website MyMiniFactory.com 1. MyMiniFac-
tory is a website that allows users to upload, share and sell 3D
models, and also provide an API to programmatically interact
with the 3D model database. Between June 10th and June 15th
2020, the API was queried using a Python script with the search
terms ‘spare part’, ‘3d printer’, and ‘accessibility’. Addition-
ally, a filter was applied to restrict results to only those under
a Creative Commons free to re-use license. A total of 1,514
files were downloaded, consisting predominantly of STL files
(1,112 files) and a range of CAD files. A small number of those
1,112 STL files were corrupted and removed, leaving a data set
of 1,049 usable files.
3.2. Surrogate model curation
Where prior work has used relatively small datasets (<100
models), this work leverages surrogate models based on exist-
ing CAD geometry to enable CNN use in the design scenario.
The open source 3D computer graphics software Blender
(version 2.8) was used to create photo-realistic renders of
the artefacts. Blender is widely used throughout the computer
games and film industry to create and render 3D graphics and
animation. It also provides a Python library, which in the case
of the work presented in this paper, allowed for scripted model
loading, creation of lighting, camera positions, and image ren-
dering.
Fixed scene lighting was used for all models with two lamps
positioned (xyz) at (L1) 4.0, 4.0, 4.0 and (L2) -4.5, -4.5, -4.5,
illuminating models as per Fig. 2.
1URL: MyMiniFactory.com Last visited: 2020-11-25, Author: MyMini-
Factory
2
R. Real et al. /Procedia CIRP 00 (2019) 000–000 3
Table 1: Rendering time per artefact (RTPA), and total dataset generation time
(TDGT) for a repository size of 1000 surrogate STL models, rendered at vary-
ing degree steps (DST).
DST Renders PA (Total) RTPA (mm:ss) TDGT(dd:hh:mm)
10684 (684,000) 06:00 04:04:00
3084 (84,000) 02:30 01:17:40
6024 (24,000) 00:12 00:03:20
9012 (12,000) 00:06 00:01:40
1206 (6000) 00:03 00:00:50
𝑥
𝑦
𝑧
𝛼
𝛽
Fig. 1: Camera sphere (From: [4])
Model size was also normalised and scaled via re-scaling of
the bounding box, such that the artefact could be represented
fully in a 540px x 540px rendered output.
Camera positions were set using longitude and latitude an-
gles, allowing the camera to be rotated around the model (see
Fig. 1). To generate a range of images the longitude and latitude
angles were set at 10, 30, 60, and 120degrees, resulting in
four data sets of 684, 84, 24 and 6 renders/images per artefact
respectfully. A summary is provided in Table 1.
3.3. CNN selection
This study utilised an AlexNet ‘o-the-shelf’ CNN archi-
tecture. The network, designed for a 1000 category classifica-
tion capacity, established prominence in computer vision re-
search with a breakthrough Top-5 performance in the 2012
ILSVRC contest, significantly outperforming prior architec-
tures. Through its substantial documentation in literature and
relative ease of use, AlexNet remains a widely used CNN ar-
chitecture with diverse applications. Specifically to this context,
[4] showed AlexNet to outperform other top classification ar-
chitectures, GoogleNet, and ResNet, achieving 94.9% accuracy
Fig. 2: Sample synthetic renders at 30 Degree view representations
with a six class surrogate model training data set when tested
against photographs of the physical artefact.
AlexNet’s architecture consists of 62 million parameters and
a 1000-way output classifier [8]. It comprises eight layers; the
first five are convolutional and remaining three layers are fully
connected (Fig. 3). The final layer, a 1000-way softmax classi-
fier, outputs a probability distribution across 1000 class labels.
The network input is a 3 channel RGB image of dimensions
227 x 227 pixels. To reduce over-fitting, complexity measures
including dropout and data augmentation are introduced. The
CNN was recreated using TensorFlow 2.0 and Keras deep learn-
ing API. A Nvidia 8GB RTX 2060 GPU was used for training.
3.4. CNN preparation
Three steps were taken to prepare the CNN following guid-
ance in literature and early experimentation with the surrogate
data. These are:
Forming the data pipeline and applying image augmenta-
tion prior to parsing through the CNN.
Configuring the Hyperparameters.
Training the CNN.
3.4.1. Data pipeline and Augmentation
From the Keras data preprocessing utilities a pipeline was
defined to generate batches of tensor image data with real-time
augmentation. The CNN was fed images from the surrogate
data set using flow from directory method, automatically in-
ferring class (artefact name) from the sub-directory structure.
30% of images per class were partitioned to a validation subset
for training. Additionally, parallel image data generators were
created for both training and validation data subsets, allowing
training data to be generated with augmentation and data shuf-
3
R. Real et al. /Procedia CIRP 00 (2019) 000–000 4
Fig. 3: AlexNet network architecture, containing five convolutional and three fully-connected layers with a 1000-way SoftMax output.
Fig. 4: Synthetic Data generation pipeline, with generator (a) augmenting im-
age outputs.
fling whilst preserving artefact representations in the validation
generator [15]. This process is shown in figure 4.
3.4.2. Hyperparamaters
To improve training stability and CNN generalisation, a
mini-batch size of 32 was used to estimate the error gradi-
ent [9]. Respectively, a learning rate of 1e-06 was applied to
scale the value by which weights were updated in back propa-
gation. These values were found to be eective with batch size,
and the generated synthetic image data. An epoch count of 200
was defined to observe training performance over time and al-
low sucient space for model convergence.
3.4.3. Training
CNN parameters are trained with weights and biases ini-
tialised on the active data set. ‘Adam’ is used as the optimisation
algorithm, and categorical cross entropy set as the loss func-
tion [7]. This approach is elected over transfer learning with
pre-trained networks to explore causality between our surro-
gate model data structures and the CNN’s performance. Addi-
tionally, hyperparameters are kept constant across data sets and
training iterations.
4. Evaluation
CNN performance was evaluated using Keras metrics on
three of five generated data sets (30, 60, 90), measuring clas-
sification accuracy and loss for training and validation steps,
Top-5 categorical accuracy, and training time. Initial test data
showed performance to be inconclusive for the 120data set,
and the 10data set to be computationally inecient, therefore
these were omitted from the study. Results from the candidate
data sets were logged and graphed to visualise performance
across dimensions of interest; accuracy, number of classes,
model coverage (number of Images per class), and training
time.
5. Results
This section reports the results from our study into a CNN
trained on a CAD surrogate model dataset. These have been
reported with respect to training performance and classification
accuracy.
5.1. Training Performance
Training time (Table 2) shows that higher quantities of arte-
fact in a given training set correlate to longer CNN training
times. Time is observed to increase exponentially across data
sets, however the degree reached by training time is limited by
data set size.
A uniform transition exists from exponential to linear growth
in training time for each of the surrogate data sets. The 30set
(featuring the highest number of training images) transitions to
more linear time scaling earlier in the class range (200 classes),
whilst this behaviour is observed later (400 classes) in the 60,
and 90data sets; suggesting the experimental set-up to reach a
performance cap at between 16,000, and 4800 images.
4
R. Real et al. /Procedia CIRP 00 (2019) 000–000 5
Table 2: Number of classes per data set and model training times
Training Time (hh:mm)Avg Epoch Time (mm:ss)
Number of Classes 306090306090
5 00:18 00:05 00:04 00:05 00:01 00:01
10 00:35 00:10 00:05 00:10 00:03 00:01
25 01:24 00:25 00:11 00:25 00:07 00:03
50 02:37 00:45 00:23 00:47 00:13 00:07
100 05:07 01:30 00:43 01:33 00:26 00:13
200 10:13 03:00 01:30 03:02 00:52 00:27
400 22:15 05:40 02:55 06:41 01:42 00:53
600 33:10 08:30 04:30 09:58 02:36 01:18
800 44:30 11:40 05:40 13:23 03:24 01:44
1000 55:07 14:20 07:20 16:33 04:12 02:06
5.2. Classification accuracy
In contrast to training time, classification accuracy decays
almost exponentially as the number of classes per data set in-
creases, showing a positive correlation between training image
quantity and classification accuracy. Thus, accurate classifica-
tion of physical artefacts can only be achieved with a small
number of classes.
Top-5 performance (by which the true artefact is named in
the top 5 predictions) displays an anity to the slope of Top-1
accuracy, although its gradient indicates a slower decay in clas-
sification performance and is more linear in nature. At 1,000
classes, the CNN was still able to classify the matching model
in the Top-5 predictions 75% of the time. This is a promising in-
dication that CNNs could support design information search &
retrieval applications. It’s worth noting for illustrative purpose
that the average car features 30,000 components.
6. Discussion and future work
This study has explored the performance properties of CNNs
trained on surrogate models for future application to Engineer-
ing Design. Particularly, this work has shown the potential of
adapting existing ‘o-the-shelf’ CNN architectures to support
Design activities. This is of significance as it demonstrates
that engineering organisations can use existing, general purpose
AI/ML architectures rather than requiring expert consultancies
to develop bespoke solutions.
Results have shown that CNNs could be applied to auto-
mated twinning between digital-physical assets if the number
of artefacts to be classified remains low (<10). However, CNN
performance can quickly deteriorate, this should therefore be
considered in selecting suitable applications for a CNN.
Top-5 results however show stronger performance, and sug-
gest potential for CNN use as a rapid Search & Retrieval tool
for design information. Implementation could occur in a Search
& Retrieval tool that does not require bespoke multi-faceted
search strategies as is typical in engineering due to the chal-
lenges in how one describes physical artefacts, their shape and
form. This could be significant in democratising and speeding
up information search processes as design engineers are a mere
photograph away from useful search results that can take them
from the physical to the digital domain.
(a) 30
(b) 60
(c) 90
Fig. 5: Training Results
With these promising results, future work could investigate
the development of bespoke CNN architectures for the activ-
ities described. In particular, it would be interesting to apply
Information Search & Retrieval’s F-score metric to the CNN
and compare it to existing engineering design search strategies.
5
R. Real et al. /Procedia CIRP 00 (2019) 000–000 6
Optimisation in preparing and training the CNN is also an
area to explore, ensuring development of computationally ef-
ficient and sustainable architectures that support design activ-
ities. This would consider the number of renders required per
artefact and the development of a suitable scene with appropri-
ate lighting to accentuate artefact features as well as the appli-
cation of multiple scenes.
Further work into architecture performance drop-ocould
yield some interesting insights into how CNNs confuse arte-
facts, with confusion matrices providing a means to explore
this. Is it the case that the confusion matrix of a CNN would be
comparable to a similarity matrix produced by humans? This
paper has shown that there remains a wealth of research to be
done to apply Machine Learning in the domain of Design, and
further assess how it supports, or even hinders design processes.
7. Conclusion
Where prior work has shown the potential of CNNs to sup-
port Engineering Design activities, this paper has taken the next
step in this journey by examining how CNNs scale with increas-
ing class sizes of surrogate CAD models. The results have im-
portant implications in determining whether CNNs can be de-
ployed for twinning and/or information search & retrieval ac-
tivities.
The results show that existing ‘o-the-shelf’ CNN architec-
tures could be re-trained and successfully deployed to twin be-
tween physical and digital domains if the number of models
is low (<10). The results also demonstrate the potential for
CNNs to support Information Search & Retrieval activities with
the CNN being able to return a correct matching in the Top-5
for 1,000 model classes. This creates opportunity to use a sin-
gle photograph to eectively retrieve virtual models of physical
artefacts from large corpi.
Together, these results demonstrate the viability of CNN use
in a design context, the eectiveness of the novel surrogate
model training approach, and scope for future opportunities that
may be realised.
Acknowledgements
The work reported in this paper has been undertaken as
part of the Twinning of digital-physical models during pro-
totyping project. The work was conducted at the Univer-
sity of Bristol, Design and Manufacturing Futures Laboratory
(http://www.dmf-lab.co.uk) Funded by the Engineering
and Physical Sciences Research Council (EPSRC), Grant ref-
erence (EP/R032696/1). The authors would also like to thank
MiniFactory.com and their users for sharing their models.
References
[1] Barbedo, J.G.A., 2018. Impact of dataset size and variety on the eective-
ness of deep learning and transfer learning for plant disease classification.
Computers and Electronics in Agriculture 153, 46–53. doi:10.1016/j.
compag.2018.08.013.
[2] Deng, J., Berg, A.C., Li, K., Fei-Fei, L., 2010. What does classifying more
than 10,000 image categories tell us?, in: Daniilidis, K., Maragos, P., Para-
gios, N. (Eds.), Computer Vision – ECCV 2010, Springer Berlin Heidel-
berg, Berlin, Heidelberg. pp. 71–84.
[3] Gopsill, J., Goudswaard, M., Jones, D., Hicks, B., 2021. Perceptions on
shape and form, in: International Conference on Engineering Design. In
Press.
[4] Gopsill, J., Jennings, S., 2020. Democratising design through surrogate
model convolutional neural networks of computer aided design repos-
itories. Proceedings of the Design Society: DESIGN Conference 1,
1285–1294. doi:10.1017/dsd.2020.93.
[5] Hansen, C.A., ¨
Ozkil, A.G., . From Idea to Production: A Retrospective and
Longitudinal Case Study of Prototypes and Prototyping Strategies. Journal
of Mechanical Design , 031115doi:10.1115/1.4045385.
[6] Jones, D., Snider, C., Nassehi, A., Yon, J., Hicks, B., 2020. Characterising
the Digital Twin: A systematic literature review. CIRP Journal of Manufac-
turing Science and Technology 29, 36–52. doi:10.1016/j.cirpj.2020.
02.002.
[7] Kingma, D.P., Ba, J.L., 2015. Adam: A method for stochastic optimiza-
tion, in: 3rd International Conference on Learning Representations, ICLR
2015 - Conference Track Proceedings, International Conference on Learn-
ing Representations, ICLR. arXiv:1412.6980.
[8] Krizhevsky, A., Sutskever, I., Hinton, G.E., . ImageNet Classification with
Deep Convolutional Neural Networks. Technical Report. URL: http:
//code.google.com/p/cuda-convnet/.
[9] Masters, D., Luschi, C., . REVISITING SMALL BATCH TRAIN-
ING FOR DEEP NEURAL NETWORKS. Technical Report.
arXiv:1804.07612v1.
[10] Maturana, D., Scherer, S., 2015. Voxnet: A 3d convolutional neural net-
work for real-time object recognition, in: 2015 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), IEEE. pp. 922–928.
[11] Peng, X., Sun, B., Ali, K., Saenko, K., . Learning Deep Object Detectors
from 3D Models. Technical Report.
[12] Sarkar, K., Varanasi, K., Stricker, D., . Trained 3D models for CNN based
object recognition. Technical Report.
[13] Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E., . Multi-view Con-
volutional Neural Networks for 3D Shape Recognition. Technical Report.
URL: http://vis-www.cs.umass.edu/mvcnn.
[14] Ulrich, K.T., 2003. Product design and development. Tata McGraw-Hill
Education.
[15] Wang, J., Perez, L., . The Eectiveness of Data Augmentation
in Image Classification using Deep Learning. Technical Report.
arXiv:1712.04621v1.
[16] Wynn, D.C., Eckert, C.M., 2017. Perspectives on iteration in design and
development. Research in Engineering Design 28, 153–184.
[17] Zaki, H.F., Shafait, F., Mian, A., 2016. Modeling 2d appearance evolution
for 3d object categorization, in: 2016 international conference on digital
image computing: Techniques and applications (DICTA), IEEE. pp. 1–8.
[18] Zhang, X., Jia, N., Ivrissimtzis, I., 2020. A study of the eect of the il-
lumination model on the generation of synthetic training datasets URL:
http://arxiv.org/abs/2006.08819,arXiv:2006.08819.
6
... The result is users do not need to describe the shape, form or type of an object where language and lexicons can become a barrier. ML as an information search and retrieval tool has also been evidenced by Real et al., (2021) who revealed that as one increases the number of model classifications (≥ 1000), the CNN's accuracy for an exact matchsuitable for twinning -diminished but it's top-10 ranking would often include a suitably matched model. The examples provided here show that the application of ML in Engineering Design is still exploratory and at the design science stage. ...
Article
Full-text available
Classifying shape and form is a core feature of Engineering Design and one that we do this instinctively on a daily basis. Matching similar components to then reduce unique component counts, determining whether a competitors design infringes on copyright and receiving market feedback on product styling are all examples where shape and form comes into play. However, shape and form can be perceived in different ways from purely mathematical (e.g. shape grammars) to wholly subjective (e.g. market feedback) and these perceptions may not entirely agree. This paper examines the mathematical and human perceptions of shape and form through a study of classifying shapes that have been interpolated between one another, and in doing so, highlights the disparity in perceptions. Following this, the paper demonstrates how the emergent field of Machine Learning can be applied to capture mathematical and human perceptions of shape and form resulting in a means to twin this feedback into product development.
Article
Full-text available
The capability to manufacture at home is continually increasing with technologies, such as 3D printing. However, the ability to design products suitable for manufacture and use remains a highly-skilled and knowledge intensive activity. This has led to ‘content creators’ providing vast repositories of manufacturable products for society, however challenges remain in the search & retrieval of models. This paper presents the surrogate model convolutional neural networks approach to search and retrieve CAD models by mapping them directly to their real-world photographed counterparts.
Article
Full-text available
While there has been a recent growth of interest in the Digital Twin, a variety of definitions employed across industry and academia remain. There is a need to consolidate research such to maintain a common understanding of the topic and ensure future research efforts are to be based on solid foundations. Through a systematic literature review and a thematic analysis of 92 Digital Twin publications from the last ten years, this paper provides a characterisation of the Digital Twin, identification of gaps in knowledge, and required areas of future research. In characterising the Digital Twin, the state of the concept, key terminology, and associated processes are identified, discussed, and consolidated to produce 13 characteristics (Physical Entity/Twin; Virtual Entity/Twin; Physical Environment; Virtual Environment; State; Realisation; Metrology; Twinning; Twinning Rate; Physical-to-Virtual Connection/Twinning; Virtual-to-Physical Connection/Twinning; Physical Processes; and Virtual Processes) and a complete framework of the Digital Twin and its process of operation. Following this characterisation, seven knowledge gaps and topics for future research focus are identified: Perceived Benefits; Digital Twin across the Product Life-Cycle; Use-Cases; Technical Implementations; Levels of Fidelity; Data Ownership; and Integration between Virtual Entities; each of which are required to realise the Digital Twin.
Article
Full-text available
The problem of automatic recognition of plant diseases has been historically based on conventional machine learning techniques such as Support Vector Machines, Multilayer Perceptron Neural Networks and Decision Trees. However, the prevailing approach has shifted to the application of deep learning concepts, with focus on Convolutional Neural Networks (CNNs). In general, this kind of technique requires large datasets containing a wide variety of conditions to work properly. This is an important limitation, given the many challenges involved in the construction of a suitable image database. In this context, this study investigates how the size and variety of the datasets impact the effectiveness of deep learning techniques applied to plant pathology. This investigation was based on an image database containing 12 plant species, each presenting very different characteristics in terms of number of samples, number of diseases and variety of conditions. Experimental results indicate that while the technical constraints linked to automatic plant disease classification have been largely overcome, the use of limited image datasets for training brings many undesirable consequences that still prevent the effective dissemination of this type of technology.
Article
This study explores the value of visualizing the prototyping activities in a new product development process from idea to production. Through a case study of a hardware startup, we present a visualization of their prototyping processes, from early idea to the introduction of several product generations to market. We call this visualization technique ProtoMapping. Analyzing qualitative and quantitative parameters of the ProtoMap allows us to understand how prototyping strategies can change over the course of a product development process. Specifically, we focus on the prototyping of parallel concepts, iterations of concepts, manufacturing processes used for prototypes, prototype media, prototype tests, as well as prototyping of isolated or integrated systems. Based on this longitudinal analysis, we present a number of insights that are otherwise difficult to extract. We observe how parallel prototypes of isolated systems can be used to explore the solution space and later be synthesized into prototypes for integrated systems. Furthermore, we understand how the need to scale up production can lead to increased prototyping activities, as several parts of a product must be redesigned. This study illustrates the utility of a retrospective visualization approach and serves as the first step in formulation of generalized guidelines for prototyping strategies for holistic product development, through further case studies.
Article
In this paper, we explore and compare multiple solutions to the problem of data augmentation in image classification. Previous work has demonstrated the effectiveness of data augmentation through simple techniques, such as cropping, rotating, and flipping input images. We artificially constrain our access to data to a small subset of the ImageNet dataset, and compare each data augmentation technique in turn. One of the more successful data augmentations strategies is the traditional transformations mentioned above. We also experiment with GANs to generate images of different styles. Finally, we propose a method to allow a neural net to learn augmentations that best improve the classifier, which we call neural augmentation. We discuss the successes and shortcomings of this method on various datasets.
Conference Paper
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif- ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implemen- tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called dropout that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry