The UAV technology initially developed for military purposes, but in recent years, its usage diversified into various functions in both the private and public sector. The usability is increasing due to the effect of fast delivery and cost saving. The UAVs considered as an emerging technology with many potential applications in the fields of civil engineering. By combining the application of UAV and digital image analysis technique, a new structural inspection system is able to detect and analyse defects such as the crack degradation factor of concrete structures.
Bridge inspection process
In a bridge inspection process, the following aspects are observed:
- One can use UAVs in the field during bridge inspections safely. Based on the UAVs size, weight, controllability, and built-in fail-safes, the risk to inspection personnel and the public is very low.
- Defects such as surface cracks identified and viewed with a level of details equivalent to a close-up photo.
The cable stay system that consists mainly of the pylon, cables, and anchorages has to be regularly inspected to ensure its safety. For a long main span bridge such as the Sg. Johor bridge, the height of the pylon can reach up to 120 meters above the deck. The high tower causes a significant problem with the inspection process, resulting in a high cost and safety issue to the inspectors. Considering this issue, the use of UAV (Unmanned Aerial Vehicle) highly recommended. Since 2015, the use of drones in bridge inspection have been implemented in some advanced countries particularly in the U.S. and Europe. In Malaysia, the short trial inspection of pylon top and stay cable successfully performed in mid-2016 at the Sg. Johor cable-stay bridge using the Airbotix X6 drone.
The FAA (Federal Aviation Administration of U.S.) defines the UAV as an aircraft flown with no pilot on board (Zink & Lovelace 2015). The UAVs sometimes referred to as drones and the name used interchangeably. Unmanned Aerial System (UAS) is also a term that commonly used. The vehicle is controlled either autonomously or with the utilization of a remote control by a pilot from the ground. It can carry a broad range of imaging technologies including still, video and infrared sensors. In the meantime, the FAA also encourages registration for all drone operators.
Classification of UAV according to weight
By the European standard, a large UAV is greater than 150kg, small UAV is less than 150kg, and a model UAV if less than 35kg. Also, a model aircraft has following aspects:
- Engine displacement is 50CC or less
- Applications of recreation, sports, and leisure
- Only fly in the line of sight
- Prohibited aboard an organism
Kim et al. (2015) stress that a multi-rotor UAV with three of more multiple rotors mounted has been determined to be most appropriate for the structure inspection. Fixed-wing UAV cannot perform vertical takeoff and landing, therefore inappropriate for this purpose. The multi-rotor UAV unit equipped with propellers, each attached to a motor. The number of propellers denotes the name of type, e.g. quadcopter has four propellers, and hexacopter has six propellers. The latter provides two extra propellers as spares should any of the four ones not working.
How the UAV system works
While technologies and capabilities differ, the most standard UAVs share these general features:
Powered by rechargeable batteries.
- Controlled either autonomously or with a remote control device.
- Contain 4-8 rotors.
- Have the ability to use GPS to track location.
- Contain fail-safes such as ‘return to home’ technology.
- UAV includes a camera with both video and still image capabilities.
The UAV system consists of the flying copter with a receiver and the remote controller with a transmitter on the ground. The motion of the flying copter can either be controlled remotely by the pilot on the ground or preplanned autonomously. The radio communication coverage between the pilot and UAV varies depending on the designed UAV system, which often extends up to two km.
The video captured by the camera on UAV streamed in real time to a monitor on the ground, which usually attached and linked to the remote controller. A high-resolution camera mounted to a stabilizer or also called gimbal, which connected to the UAV. This 2-axis or 3-axis gimbal ensures the video and images produced not blurred due to the vibration of the camera during flight. For structural inspection, the camera installed should able to capture high-resolution images that clearly show the cracks, if any. The recorded videos and pictures are stored in a micro-SD placed inside the camera, which can be uploaded to a PC when the UAV landed.
Most modern UAVs equipped with GPS have a capability to fly fully autonomous. It can takeoff and landing automatically, and can set to fly home by itself (return home). The latter is important to avoid any crash should the battery run out, or radio communication between the flying UAV and remote controller on the ground is accidentally cut-off. The UAV can also pre-programmed to follow a certain well-defined route autonomously. Third-party software is also available to automatically capture images while following this pre-programmed route. Subsequently, a 2D or 3D model of the covered area can be produced from these captured images.
The latest technology has enabled the high-end UAV to fly either inside a building, below a bridge or wherever the GPS is not available. The optical and ultrasound sensors installed on the UAV and directed to the ground has enabled its stable flight even without the GPS to hold and monitor its positioning.
Advantages of UAV technology in bridge Inspection
Zink & Lovelace (2015) identified the following advantages of the UAV technology in bridge inspection process.
- UAVs with the ability to direct cameras upward and the ability to fly without a GPS signal are important features when using this technology as an inspection tool.
- UAV technology is evolving rapidly, and inspection-specific UAV features are just coming into the marketplace that will increase their effectiveness in bridge inspection, particularly its safety issues.
- In some types of inspections, a UAV has the capabilities to replace an under-bridge inspection vehicle, which provides significant savings. Another saving would come in the form of reduced or eliminated traffic control during the inspection work.
- UAVs can provide a cost effective way to obtain detailed information that not normally obtained during routine inspections.
- Infrared images of bridge decks and other components are already a common and accepted way to get information on concrete defects. UAVs can provide a very efficient way to collect infrared images of bridge decks and other elements.
- Safety risks associated traffic control, working at height and in traffic minimised with the use of UAVs.
- UAVs utilised as an effective method to determine stream or river bank conditions upstream or downstream of the bridge. It can also capture large overall aerial maps of dynamic bank erosion and lateral scour conditions.
- UAVs can provide important pre-inspection information for planning large-scale inspections. Information such as clearances, rope access anchor points and general conditions can easily be obtained with a UAV and would aid in the planning of an inspection.
Limitation of the UAV technology
- Short battery life – each flight usually limited to about 15 to 20 minutes depending on the payload and wind condition before replacing the onboard battery. The more expensive UAV provides provision for the installation of two batteries on the UAV, which usually double the flight duration. To reduce electrical power consumption and improve battery life, the designed UAV and the mounted camera and gimbal have to be as light as possible.
- Vulnerable to wind and rain– The UAV, especially the smaller ones, could not withstand the high wind. The high-end UAV can typically sustain the wind up to 40mph. Meanwhile, the rain would damage the expensive instruments such as the camera onboard.
- The UAV communication system has to facilitate a diverse range of camera, the installation of which shall be easy and swift. Also, the camera main functions can be controlled remotely from the ground. Presently, most UAV manufacturers designed a system that can only communicate with a specific type of camera, which mostly not suitable for bridge inspection. This system limits the choice of available UAV for the intended purpose. To overcome this shortfall, a specially written application interface (API) and other related software required to facilitate the installation of a camera not initially designed to work with the UAV system.
- Images can estimate measurements, but UAVs cannot replicate physical functions (e.g. cleaning, sounding, measuring, and testing) equivalent to a hands-on visual inspection.
- For safety reason, a UAV is not allowed to fly within 5 miles radius of an airport
- The maximum height allowed by most aviation authority is 120 meter to avoid other flying objects such as a helicopter.
Main criteria in UAV selection for crack detection
- Provide stability to withstand the high wind especially at high level near the sea where the wind is usually quite strong
- Capability to carry a quite heavy (almost one kg) high-resolution DSLR camera, e.g. 30MP with full frame sensor. One can use a lower pixel camera of 16 or 20MP with a zoom lens
- Facilitate the installation of high-resolution thermal camera, e.g. 640×512
- The inspection of the underside of the bridge deck often requires the camera mounted at the top of UAV. Therefore, the UAV has to facilitate this changing process of the camera mount can easily and swiftly perform at the site.
- Provision for the installation of other instruments such as humidity and temperature sensors on the UAV
Comparison of selected UAVs
The process to identify and select a suitable UAV for the inspection of a bridge stayed cable system is not easy considering the limited number of affordable UAV to perform this task.
There are many UAVs currently on the market with GPS and imaging capability starting at around RM5000. The full range in price attributed to features and length of battery life. A lower cost UAVs battery may only last 10-20 minutes. Meanwhile, a higher end UAV typically lasts between 40 to 60 minutes with an option to install an additional battery onboard. Lower end models typically lack post-processing software. They also have a lower material and build quality, and the inability to mount high-resolution camera. These negative points make them less suitable as a tool for bridge inspection. Nonetheless, the costs and features available on UAVs are changing rapidly as the technology advances.
Most commercially available UAVs designed for the filming of aerial view and mapping of ground profile. Since the visual inspection of bridges usually involves the detection of defects in the form of cracks, a high-resolution digital and thermal camera is necessary. A digital camera detects surface cracks while thermal camera could detect defects beneath the surface. Therefore, the two types of camera complement each other in exposing the defects of a structure.
Due to the effect of strong wind at a high level, the inspection at the top of a pylon is considerably risky. The UAV could hit the pylon and crash if flying too close to the pylon or stay cable. A minimum distance of 3 meters is often considered as safe enough for most UAVs. Thus, we recommend a high-resolution camera with adequate zoom capability that the pilot can control and adjust remotely.
Most digital cameras with the built-in 30MP sensor are considered sufficient for this purpose. But these digital cameras are often the DSLR type that is bulky and cumbersome. The heavy payload results in reduce flight time of the UAV as it consumes more energy to fly. Fortunately, there is a new technology called mirror-less camera that is considerably smaller and lighter than the DSLR camera but produces almost an equivalent resolution. Considering these facts, the mirror-less camera is the best option to detect the surface cracks successfully.
Thermal camera on a drone
For a thermal camera, size and weight are not an issue because it is considerably small and light,
irrespective of its resolution values. For example, the three variations of camera manufactured by the Flir with resolution value of 160, 320 or 640 are almost of the same size. Flir considered as the leader in thermal camera technology. Nonetheless, the price of the highest resolution thermal camera is about RM60,000. This is more expensive than the mirror-less camera such as Sony a7Rii, which has 42MP resolution. It only costs about RM12,000 without the lens.
The next step is the selection of a suitable UAV to work with the identified camera. Most UAVs capable of mounting this type of camera are considerably expensive. They usually in the range of tens thousands to a few hundred thousands Malaysian Ringgit.
The DJI is the world leader in commercial UAV manufacturer and therefore include in the selection process. Two versions of UAV identified: the Inspire 1 with the built-in camera and the Matrice M100 without a camera. The camera mounted on the Inspire 1 is of average resolution (12MP for X3 and 16MP for X5) without zooming capability. Therefore, unsuitable for surface cracks detection of a bridge.
In early 2017, Inspire 2 made available as a significant upgrade to the Inspire 1. It is considerably more expensive with the additional options for high-resolution of the micro four third (M4/3) cameras. There are two cameras available. The first is the Zenmuse X4S (US$599). It has the same 1-inch 20MP image sensor, 24mm f/2.8-11 lens, and mechanical shutter as the integrated camera used by the Phantom 4 Pro. Its fixed field of view covers about the same angle as a 24mm lens on a full-frame camera system.
The second camera option is Zenmuse X5S, (US$1899), which supports up to 10 different lens, including zooms. It also captures videos up to 5.2K quality in CinemaDNG, and can shoot 20MP stills in DNG and JPG formats. The DNG Raw mode allows 30MP still images.
Unlike Inspire 1, this UAV has the fixed forward camera for transmitting the FPV image. This camera enables the pilot to send the drone flying in a certain direction while recording footage from another angle. With dual controller available, this function is great especially for filming video or movie.
With two batteries on the Inspire 2 it can fly for longer than the Inspire 1. According to DJI the operation time can reach 27 minutes on the Inspire 2, compared to 18 minutes on the Inspire 1.
Meanwhile, the Matrice M100 is an open concept DIY type. It has the software development kit (SDK) readily available for downloading at DJI website. Suitable for research and development purpose, this UAV could provide an answer for mounting a larger high-resolution camera.
However, an expert knowledge required on a suitable gimbal to hold the camera on the UAV. With the optional camera and gimbal installed, the drone has to balance properly. Otherwise, it can easily crash during flight. Also, the development of software required to ensure a seamless integration of the camera and gimbal within the system.
Another shortfall of this UAV is the allowable payload is only one kg. This is inadequate to carry a large camera with a gimbal, together with the hardware for the collision avoidance system. For carrying heavy payload, one has to opt for a more expensive commercial version M200 series (up to 2kg) or M600 Pro (up to 6kg).
Collision avoidance system of a drone
Equipped with a sophisticated collision avoidance system as an option, this type of UAV seems to be an excellent choice. The DJI claims to be the first of such a system in the world. The avoidance system could detect obstructing object within the range of 0.2m to 30m at all four sides and below.
For example, the Inspire 2 has forward and downward vision systems. Even travelling at 54kph, it can detect obstacles up to 30 meters ahead and avoids it. Also, the upward facing infrared sensors can scan obstacle 5m above. This capability adds protection when flying underneath a structure such as a bridge.
Other drone options
We have identified and tabulated a few UAVs for comparison of their important criteria in the selection as shown in Table 1.
Table 1. Comparison of UAVs
|Manufacturer/Model||Weight (kg)||Flight time (minutes)||Camera type||Price (USD)|
|DJI Inspire 1 Pro||3.0||18||Zenmuse X5||3,400|
|DJI Inspire 2||3.3||27||Zenmuse X5S||4,900|
|DJI Matrice M100||2.4||35||none||4,300|
|Aibotix Aibot X6||2.6||20||none||20,000|
The use of drones or UAVs to aid bridge inspections should consider for routine inspections to improve the quality of the inspection. These UAVs provide information and details that may not available without expensive access methods. The clients should also consider using the UAV system where it increases safety for inspection personnel and the traveling public.
This emerging technology should also apply in other aspects of civil engineering inspection processes such as the slope stability evaluation, dams safety monitoring and tunnel lining maintenance.
Zink, J. and B. Lovelace (2015). Unmanned Aerial Vehicle Bridge Inspection Demonstration Project: Final Report, Minnesota Department of Transportation, U.S.
Kim, J.W., S.B. Kim, J.C. Park and J.W. Nam (2015). Development of Crack Detection System with Unmanned Aerial Vehicles and Digital Image Processing, World Congress on Advances in Structural Engineering and Mechanics (ASEM15), Incheon, Korea
The author, Dr. Ab Nasir Jaafar is a practising civil engineer who involved in the maintenance of a few cable-stay bridges in Malaysia.