Field imaging framework for morphological characterization of aggregates with computer vision: Algorithms and applications

Huang, Haohang

Permalink

https://hdl.handle.net/2142/117876 Copy

Description

Title

Field imaging framework for morphological characterization of aggregates with computer vision: Algorithms and applications

Author(s)

Huang, Haohang

Issue Date

2021-12-03

Director of Research (if dissertation) or Advisor (if thesis)

Tutumluer, Erol

Doctoral Committee Chair(s)

Tutumluer, Erol

Committee Member(s)

• Al-Qadi, Imad L.
• Roesler, Jeffery R.
• Golparvar-Fard, Mani
• Patel, Sanjay

Department of Study

Civil & Environmental Eng

Discipline

Civil Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Construction Aggregates
• Riprap
• Field Imaging
• Morphological Characterization
• Stockpile Analysis
• Computer Vision
• Computer Graphics
• Artificial Intelligence
• Deep Learning

Abstract

Construction aggregates, including sand and gravel, crushed stone and riprap, are the core building blocks of the construction industry, national economy, and society. In the year 2020, in total 2.42 billion metric tons of aggregates valued at 27.0 billion were produced by about 5,400 mining companies operating more than 10,000 quarries across all 50 states. Through mining, quarrying, and multi-level crushing and screening processes, aggregates produced in different sizes and forms constitute the main skeleton of civil infrastructure and are extensively used in structural, transportation, geotechnical, and hydraulic engineering applications. At both quarry production lines and construction sites, the morphological properties of aggregates (such as size, shape, volume/weight, etc.) are some of the most crucial indicators for aggregate Quality Assurance and Quality Control (QA/QC), especially for crushed aggregates and riprap. State-of-the-practice methods mainly use sieving and caliper devices for the size and shape determination of the most commonly used regular sizes of crushed aggregates, and are limited to visual inspection and manual measurement for relatively large-sized aggregates. As a more advanced quantitative approach, state-of-the-art aggregate imaging methods developed to date focus on characterizing aggregate morphology from acquired image data and machine vision analysis, yet with the limitation that most systems are only applicable to regular-sized aggregates under well-controlled laboratory conditions. The state-of-the-practice and state-of-the-art methods have encountered several major challenges in characterizing aggregate morphology. First, quantitative methods for capturing and analyzing aggregates are required to provide reliable characterization of the material. Second, flexible and effective methods are urgently needed for relatively large-sized aggregates. Furthermore, advanced analyses are necessary to handle the most practical form of aggregate presence, such as densely stacked aggregates in stockpiles and/or in constructed layers. Lastly, three-dimensional (3D) imaging approaches are deemed ideal by providing more comprehensive and realistic aggregate information than two-dimensional (2D) image analyses. This dissertation presents the research effort to address these major challenges by developing a field imaging framework for the morphological characterization of aggregates as a multi-scenario solution. The framework also has a focus on relatively large-sized aggregates, for which effective and efficient field characterization methods are extremely lacking. For individual and non-overlapping aggregates, a field imaging system was designed first, and the associated image segmentation and volume estimation algorithms were developed. The color-based image segmentation algorithm provides robust object extraction under various field lighting conditions such as strong sunlight and shadowing, and the volumetric reconstruction algorithm estimates the particle volume by orthogonal intersection. The approach demonstrated good agreements with ground-truth measurements made at quarry sites and achieved great improvements in the volumetric estimation of individual aggregates when compared with the state-of-the-practice inspection methods. For 2D image analyses of aggregates in stockpiles, an automated 2D instance segmentation and morphological analysis approach was established based on deep learning. A task-specific stockpile aggregate image dataset was compiled based on images collected from aggregate producers and individual aggregates in the images were manually labeled to provide the ground-truth for learning. A state-of-the-art object detection and segmentation architecture was implemented to train the image segmentation kernel for stockpile segmentation. The segmentation results showed good agreement with ground-truth labeling and provided efficient morphological analyses on images containing densely stacked and overlapping aggregates. For 3D point cloud analyses of aggregates in stockpiles, an end-to-end, integrated 3D Reconstruction-Segmentation-Completion (RSC-3D) approach was established by collaborating three developed components, i.e., laboratory and field 3D reconstruction procedures, 3D stockpile instance segmentation, and 3D shape completion. The approach was designed to reconstruct aggregate stockpiles from multi-view images, segment the stockpile into individual instances, and predict the unseen side of each instance based on the partial visible shapes. First, a 3D reconstruction procedure was developed to obtain high-fidelity full 3D models of collected aggregate samples, based on which a 3D aggregate particle library was constructed, and a comparative analysis was conducted regarding the 2D and 3D morphological characteristics. Next, two datasets were prepared based on the 3D particle library for 3D learning purpose: (i) a synthetic dataset of aggregate stockpiles with ground-truth instance labels developed with a synthetic data generation pipeline involving model fabrication, stockpile assembly, and stockpile raycasting; and (ii) a dataset of partial-complete shape pairs, developed with varying-visibility and varying-view raycasting schemes. Based on the two datasets, a state-of-the-art 3D instance segmentation network and a 3D shape completion network were implemented and trained, respectively. The application of the integrated approach was demonstrated on re-engineered stockpiles and field stockpiles, and the validation results against ground-truth measurements showed good performance in capturing and predicting the unseen sides of aggregates, especially in terms of size dimension metrics. In summary, the developed field imaging framework in this study encompasses three major approaches that characterize various forms and representations of field aggregates with increasing analysis complexity: (i) a volumetric reconstruction approach for individual and non-overlapping aggregates; (ii) a 2D instance segmentation and morphological analysis approach for aggregates in stockpiles based on 2D image analysis; and (iii) a 3D integrated reconstruction-segmentation-completion approach for aggregates in stockpiles based on 3D point cloud analysis. The framework addresses the major challenges of characterizing individual aggregates and aggregate stockpiles in the field, thus provides a multi-scenario solution for efficient 2D and 3D analyses of aggregates.

Graduation Semester

2021-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2021 Haohang Huang

Owning Collections