Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Applications of Deep Learning Techniques for Pedestrian Detection in Smart Environments: A Comprehensive Study

Applications of Deep Learning Techniques for Pedestrian Detection in Smart Environments: A... Hindawi Journal of Advanced Transportation Volume 2021, Article ID 5549111, 14 pages https://doi.org/10.1155/2021/5549111 Review Article Applications of Deep Learning Techniques for Pedestrian Detection in Smart Environments: A Comprehensive Study 1 2 3 4 Fen He , Paria Karami Olia , Rozita Jamili Oskouei , Morteza Hosseini , 5 2 Zhihao Peng , and Touraj BaniRostam School of Information Engineering, Guangzhou Nanyang Polytechnic College, Guangzhou, Guangdong, China Computer Engineering Department, Islamic Azad University, Central Tehran Branch, Tehran, Iran Department of Computer Science and Information Technology, Mahdishahr Branch, Islamic Azad University, Mahdishahr, Iran Department of Engineering, Islamic Azad University, South Tehran Branch, Tehran, Iran School of Software, Dalian Neusoft University of Information, Dalian, China Correspondence should be addressed to Rozita Jamili Oskouei; rozita2010r@gmail.com Received 8 February 2021; Revised 1 April 2021; Accepted 24 August 2021; Published 4 October 2021 Academic Editor: Chunjia Han Copyright © 2021 Fen He et al. )is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Intelligent transportation systems have been very well received by car companies, people, and governments around the world. )e main challenge in the world of smart and self-driving cars is to identify obstacles, especially pedestrians, and take action to prevent collisions with them. Many studies in this field have been done by various researchers, but there are still many errors in the accurate detection of pedestrians in self-made cars made by different car companies, so in the research in this study, we focused on the use of deep learning techniques to identify pedestrians for the development of intelligent transportation systems and self- driving cars and pedestrian identification in smart cities, and then some of the most common deep learning techniques used by various researchers were reviewed. Finally, in this research, the challenges in each field are discovered, which can be very useful for students who are looking for an idea to do their dissertations and research in the field of smart transportation and smart cities. driver is drowsy or the driver does not pay attention to 1. Introduction hazards to prevent accidents. In recent years, intelligent transportation systems have been Some of the most important problems in creating and developed to help reduce the volume of traffic in metro- developing self-driving cars are as follows: politan areas, reduce the rate of accidents and injuries and (i) Lack of accurate patterns to identify pedestrians deaths caused by them, reduce fuel consumption, reduce and roadblocks with very high accuracy in different environmental pollution, and so on. )ese systems use roads with different light intensities and different different technologies (including IoT, machine learning and image quality data mining, neural networks, deep learning, and image processing) for various applications. On the other hand, (ii) Error in identifying pedestrians and obstacles in the paths of self-driving cars large automotive and technology companies (such as Google and Tesla) are trying to produce self-driving smart cars that (iii) People’s distrust of self-driving cars can provide safe travel for people when the driver is drowsy (iv) Lack of acceptance of people for 24-hour control and that can save his life as well as passengers’ lives, with and monitoring of various urban and interurban automatic control car to prevent any accident. )ese vehicles roads’ paths must be equipped with sensors to sense the environment and (v) Negative effects of dark weather and snow/ice and identify objects close to the car and inform the driver and also have actuators to perform real-time operations when the fog and rain on the quality and performance of 2 Journal of Advanced Transportation cameras installed on cars, ultimately reducing the In recent years, organizations responsible for transport accuracy of pedestrian detection and obstacles in management in different countries of the world have shown these vehicles great attention to the development and use of intelligent transport systems using the creation of intercity networks. (vi) Lack of necessary infrastructure to implement in- )e main reason for this is the diverse applications of au- telligent transportation on all urban and interurban tomotive communication technology in the four areas of roads safety promotion, mobility improvement, environmental (vii) )e need for high investment to implement the protection, and asset management. Automotive intelligent necessary infrastructure to implement intelligent communication systems provide technical and economic transportation and communication systems to solutions to the transportation challenges of the 21st century. connect vehicles to each other (V2V) and vehicles )ese systems must be able to enable a growing segment of with roadside infrastructure (V2I) the human population to move freely, without the risk of Machine learning (ML) is a field of artificial intelligence that accidents and with minimal fuel consumption and envi- uses statistical techniques to learn hidden patterns from existing ronmental pollution [2]. data and to make decisions about unseen records. )e main task Intelligent transportation technology ranges from basic management systems (such as car navigation, traffic signal of a machine learner is building a general model on the possible distribution of training examples and then generalizing expe- control systems, variable driving signs, automatic license plate number recognition, or speed camera) to surveillance appli- rience to unseen examples [1]. )e learning process depends on the quality of the data displayed. An example is presented in a cations such as advanced CCTV security systems that provide information. From sources such as car park guide information dataset with different properties. Unfortunately, extracting ef- ficient features can be difficult for some tasks. systems, they collect weather information, etc. [2]. Deep learning is an advanced branch of the ML disci- Intelligent Transportation System (ITS) means the use of pline that aims to discover complex representations of a set of tools, facilities, and expertise such as traffic engi- simpler representations. Deep learning methods are usually neering concepts, software, hardware, and telecommuni- cations technologies in a coordinated and integrated manner based on artificial neural networks consisting of several hidden layers with nonlinear processing units. )e word to improve the efficiency and security of transportation systems. deep refers to several hidden layers used to change the display of data. Using the concept of feature learning, each Intelligent transportation systems can also be general- ized to different modes of transportation, in which, using hidden layer of neural networks plots its input data in a new display. )e layer manages to absorb a higher level of ab- automated tools and related scheduling, various types of straction than the abstract concept in the previous layer. In information receiving and processing operations, as well as deep learning architectures, the hierarchy of features learned traffic management and control, are performed. In this at multiple levels is finally mapped to the output of the ML system, by limiting the role of human factors in information work in a single framework. Similar to ML methods, the processing or control and management processes, we im- deep learning architecture is divided into two broad cate- prove the quality of decision-making and management processes. gories: (a) unsupervised learning methods and (b) super- vised learning approaches, including deep neural networks. In intelligent transportation systems (ITS), the definition of transportation infrastructure in addition to information )is research is organized in five sections. In the second part, we provide a complete description of intelligent and communication technologies leads to the achievement of goals such as improving passenger safety, reducing transportation systems. In the third section, we provide a brief overview of deep learning and some of its applications. transportation time, and reducing fuel consumption and In the fourth section, we describe the use of deep learning to wear or tear of car tires. Applications of ITS include accident identify pedestrians in smart cities and intelligent trans- management, electronic toll collection management, public portation systems, review some of the research conducted by transportation management, passenger communication various researchers in this field, and state the challenges in management, and traffic flow management [3]. each area of research. Finally, in the fifth section, we will Compared to traditional traffic engineering, the intelli- gent transportation system (ITS) has created a new trans- provide the conclusion. portation system. Due to different national circumstances, the development priorities of these systems are different, and 2. Intelligent Transport Systems (ITS) therefore, the content of ITS research is not the same in all countries. In general, ITS uses information, communication, Intelligent transportation systems, for automatic road man- agement and real-time operations in the event of an natural control, computer technology, and other current technol- accident (such as a mountain fall, avalanche, and icy road ogies to create a real-time, accurate, and efficient trans- floor) or unnatural disasters (such as accidents, road repairs, portation management system. and car traffic), have been developed. )ese systems use sensors as a tool to identify and understand the state of the travel environment. )e data collected by these sensors using 2.1. Architecture of Intelligent Transportation Systems. )e US Department of Transportation, through RITA Research communication technologies (such as WiFi and DSRC) to other vehicles on the route are sent to control centers. and Innovation Technology Management, defined a Journal of Advanced Transportation 3 2.2. Some Services Provided in Intelligent Transportation national architecture for ITS and provided a common structure for designing intelligent transportation systems. Systems. Some of the most common services provided in intelligent transportation systems are briefly described in )e ITS function model (Logic Architecture) provides a functional view of ITS user services. Physical architecture this section [5]. divides the functions defined by logical architecture into classes and subsystems. Figure 1 shows the high-level 2.2.1. Accident Management. We divide the stages of acci- diagram of the proposed physical architecture (Archi- dent management into five stages (depending on the type tecture Development Team 2007a), in which 22 subsystems and severity of the accident) in which one or more stages (white rectangles) are distributed among four classes: may occur simultaneously [5]: passengers, centers, vehicles, and the field or area of (i) Detection and notification: it is the detection and operation. notification of accidents that are often used in In Figure 1, the communication requirements between mobile phones at this stage. these subsystems are supported by four types of commu- nication, which are shown in the form of an oval in the (ii) Verification: the existence of the accident and the figure: wireless communication over a wide area, fixed exact type and location of the accident (via traffic point-to-fixed point communication, vehicle-to-vehicle surveillance cameras) are confirmed. communication, and dedicated short-range communica- (iii) Incident site management: it is a complex process tions [4]. that requires careful coordination, communication, )e following is a brief description of each of the classes and cooperation between the people present on the in Figure 1: scene, all supporting institutions and the general (i) Travelers: different services are provided to pas- public. Important points for proper incident scene management include the following: sengers (including drivers and occupants of cars), which are generally grouped into two categories, (i) Providing accurate information to the dispatch which are as follows [4]: unit in a timely manner, including the exact (i) Support to their remote travel: using the in- location of the accident, the severity of the stallation of surveillance cameras by the Road accident, and so on (ii) Establishing evidence in a safe area to minimize and Transportation Administration on various routes inside and outside the city, road and oncoming traffic risk given the location of damaged vehicles and rescue personnel transport managers can monitor the move- ments of passengers on different routes from the (iii) Establishing a command system, especially headquarters and in case of any problem, when major events occur whether it is an accident or a fall of a mountain, (iv) Asking for help from cleaning companies if and etc., and send a team to the place there is a possibility of hazardous substances at immediately the scene (ii) Access to personal information: it includes (v) Public information in case of an accident monitoring of the crossing of intersections through mass media, Internet, or SMS to when the traffic lights are red and registering the passengers license plate number of the offending vehicle (iv) Protection of evidence: it includes the protection of and then issuing a fine for him evidence at the scene of the accident and potential (ii) Control centers including the centers providing evidence that may later be used to prosecute the perpetrator or analyze the data in the future. Any necessary and useful information for drivers, traffic management centers, relief and emergency centers, suspicious items at the scene (such as guns, bullets, drugs, and alcohol) should also be protected to be transport management centers, toll collection cen- ter, collected data management centers, transport handed over to the police on the scene or the highway warden [5]. fleet control centers, and road maintenance management (v) Hazardous Materials: when hazardous materials (iii) Vehicles including personal vehicles, emergency are spilled at the scene of an accident, they must vehicles, commercial vehicles, freight vehicles, first be thoroughly inspected by police dis- support vehicles and relief vehicles, or vehicles patchers, and then, the necessary measures must belonging to the police patrol be taken to collect those materials and clean the environment. (iv) Roadside equipment including equipment installed (vi) Breakdown and Demobilization: public mobiliza- on the road, equipment related to toll collection, parking management, checking commercial vehi- tion to clear the scene and analyze the accident that occurs when all injured people, damaged vehicles, cles, and checking the weight of trucks with their load, which, if it is more than a certain weight, equipment, and debris are removed from the scene. should have stopped and in addition to fining them, Public mobilization is to create security, expedient, their burden should also be reduced and regular departure of all those present at the 4 Journal of Advanced Transportation Advanced Traveler ATIS Information Systems ✓ traffic information ✓ travel guidance Advanced Traffic ATMS ✓ optimized route guidance Management System ✓ Rcal-time traffic control ITS Advanced Public ✓ Incident management Intelligent Transportation Systems ✓ Automatic vehicle APTS Transport enforcement System ✓ Public tranportation ✓ Automatic Toll collection information ✓ Public traffic management Advanced Vehicle AVHS Highway System Commercial Vehicle ✓ AVS:Advanced Vehicle System CVO Operations - Vehicle Automatic Control System ✓ FFM: Freight and Fleet Management ✓ AHS: Advanced Highway System ✓ HMM: Hazard Material Management Figure 1: High-level architecture diagram for ITS [4]. scene of the accident and equipment and vehicles systems, and vehicle connectors) are combined to form a from the scene and return the affected area to cohesive interface that is capable of parsing and has real-time data analysis and decision-making about the current traffic normal with normal traffic flow [5]. situation and the assessment of subsequent conditions that may occur, as well as the adoption of appropriate measures 2.2.2. APTS Public Transportation Management. )ese to deal with the conditions that have arisen: dynamic traffic systems use new information management technologies to control systems, highway operations management systems, increase the efficiency and enhance the security of public accident prevention systems, and making necessary and transportation systems. )ese systems include instantaneous appropriate decisions when accidents occur, etc. )ey are and real-time passenger information management systems, considered as advanced traffic management systems [5]. vehicle location detection systems, bus arrival time notifi- cation systems, and bus crossing priority prioritization 2.2.5. Network Security Management. )e main purpose of systems. network safety management, somewhat like the manage- ment of sensitive points, means identifying the areas where 2.2.3. Advanced ATIS Passenger Information Systems. accidents are most likely to occur. )erefore, there is an )ese systems provide information on travel routes and urgent need to ensure road safety in those areas. However, weather conditions for transportation system users, so that there are two important differences between hotspot man- they can make the right decisions to choose the route, es- agement and network security management [5]: timate travel time, and avoid getting caught in crowded (i) In network safety management, the important goal is routes. Several technologies are used for this purpose, which to identify roads with different degrees of security and are as follows [5]: ultimately to identify accident hotspots or sensitive (i) GPS enabled in car navigation systems points in the road system (such as intersections). (ii) Dynamic signs and messages for timely and real- (ii) In network safety management, a report on the severity time notification in traffic, turns and passes, and of accidents is prepared, and accident-prone parts of accidents or when the road is closed for various the road are identified. In the management of critical or reasons such as repairs accident-prone points, the number of accidents at each critical point is usually too high, so this point is given (iii) Websites are used to indicate congestion on high- more importance than the severity of the accident. ways, main streets, and urban and interurban road networks 3. Deep Learning 2.2.4. Advanced ATMS Traffic Management Systems. )e Learning is the process by which a system improves its data and information obtained through different subsystems performance by using past experiences. Since 2006, deep (such as vehicle type identifiers, in-vehicle messaging learning has emerged as a new subfield of machine learning, Journal of Advanced Transportation 5 (i) )e training phase involves labeling large amounts affecting a wide range of signal and information processing in both traditional and modern fields. Many traditional of data and determining their adaptive properties. machine learning and signal processing techniques use (ii) )e inference step is to conclude and label new and special architectures that contain a single layer of nonlinear unseen data, using their prior knowledge. Deep features. learning is a method that helps the system under- Some examples of deep learning in the workplace in- stand the complex tasks of perception with maxi- clude a self-propelled vehicle slowing down as it approaches mum accuracy. Deep learning is also known as deep a pedestrian crossing, an ATM rejecting a counterfeit structured learning and is a hierarchical learning that banknote, and a smartphone app instantly translating an consists of several layers that include nonlinear installed signboard performing on the street. Deep learning processing units to convert and extract features. is especially suitable for identification programs such as face Each subsequent layer takes the results from the recognition, text translation, voice recognition, and ad- previous layer as input. vanced driver assistance systems, including and symptom )e learning process is performed using the distinct recognition [6]. stages of abstraction and multiple levels of representation in a supervised or unsupervised manner. Deep learning or deep neural network uses a basic computing unit, a neuron that 3.1. .e Difference between Deep Learning and Machine receives multiple signals as input. It integrates these signals Learning. Deep learning is one of the subfields of machine linearly with the weight and transmits the combined signals learning. By learning the machine, the features of an image to the nonlinear tasks to produce output. can be extracted manually. With deep learning, raw images In the “deep learning” method, the term “deep” refers to can be inserted directly into a deep neural network that the multiple layers through which data is converted. )ese learns features automatically. Deep learning usually requires systems are composed of a very special deep credit allo- hundreds of thousands or millions of images to get the best cation (CAP) path, which means that the steps were per- results, while machine learning works well with small formed to convert the input to output and represent the datasets. Deep learning is also computationally intensive and impact connection between the input layer and the output requires a high-performance CPU [7]. layer [7]. It should be noted that there is a difference be- Deep learning is the most effective, supervised, and cost- tween deep learning and machine learning. Machine effective machine learning approach. Deep learning is not a learning involves a set of methods that help the machine limited learning method, but it follows a variety of methods receive raw data as input and set views for the purpose of and topographies that can be used to make broad predictions detection and classification. Deep learning techniques are about complex problems. )is technique includes descrip- simply a type of learning method that has several levels of tive and distinctive features in a completely categorized way. representation and is at a more abstract level. Figure 2 Deep learning methods with remarkable performance have shows the difference between machine learning and deep achieved significant success in a wide range of applications learning. with useful security tools. Deep learning is used in many Deep learning techniques in large databases use non- applications, including business, comparative experiments, linear transformations and high-level model abstraction. biological image classification, computer insight, cancer )ey also describe how a machine can change the internal detection, natural language processing, object recognition, features needed to count descriptions in each layer by face recognition, handwriting, speech recognition, stock accepting abstractions and displaying previous layers. )is market analysis, and creation and the development of smart new learning approach is widely used in the areas of adaptive cities. testing, big data, cancer diagnosis, data flow, document Machine learning is a subset of artificial intelligence (AI) analysis and identification, healthcare, object recognition, that gives systems the benefits of automatically learning speech recognition, image classification, pedestrian detec- concepts and knowledge without explicit planning. It begins tion, natural language processing, and voice activity with observations such as direct experiences to prepare detection. features and patterns in the data and to produce better )e deep learning model uses a set of features set for results and decisions in the future. Deep learning relies on a large features using bulk dataset for unique features, then set of machine learning algorithms that model high-level extracts a classification model, and creates an integrated abstractions in data with multiple nonlinear transforma- classification to explore a variety of applications. tions. Deep learning technology works on an artificial neural )e key factors on which the deep learning method is network (ANN) system. )ese neural networks continuously based are as follows [8]: use learning algorithms, and by constantly increasing the amount of data, the efficiency of training processes can be (i) Nonlinear processing in multiple layers or stages: nonlinear processing in multiple layers refers to a improved. )e efficiency of deep learning algorithms de- pends on the volume of large data. )e process is called deep hierarchical method in which the present layer ac- cepts the results of the previous layer and transmits training, because the number of neural network levels in- creases over time. its output as input to the next layer. Hierarchy is )e operation in the deep learning process generally created between layers to organize the importance of the data. depends on two stages called the training and inference. 6 Journal of Advanced Transportation Machine Learning )e results of this study showed that the use of deep learning techniques in comparison with the use of data Car mining techniques both reduces the time of analysis and Not Car detection of normal and abnormal behaviors and in- Input Feature extraction Output Classification creases the accuracy of identifying abnormal behaviors of pedestrians. )e researchers pointed out that modeling Deep Learning pedestrian behaviors and behavioral analysis in the de- velopment of smart cities can help increase the efficiency Car of smart agents used for various applications in these Not Car cities. Input Feature extraction • Classification Output Figure 2: )e difference between machine learning and deep Challenge. )e limited features used to model pedestrian learning [8]. behaviors and the need to apply metaheuristic algorithms to solve complex intelligent computing problems are among the major challenges in this research to model pedestrian (ii) Supervised or unsupervised learning: here, supervised behaviors. and unsupervised learning are linked to the class goal Kim et al. [10] examined pedestrian identification in label. Its availability means a supervised system, and smart buildings. Because the identification of pedestrians its absence indicates an unattended system. due to noise in images and some environmental factors and parameters faces challenges. )e researchers used the Deep 4. Using Deep Learning to Diagnose Pedestrians Convolution Neural Network (CNN) to create a vision- based model and the optimized version of the VGG-16, In today’s world, where the development of smart cities and called the OVGG-16, as the architectural core used to dis- smart transportation has received a lot of attention from tinguish pedestrians from the multitude of possible images. people, governments, and commercial and manufacturing To evaluate the proposed method, the researchers used the companies, one of the basic needs is to provide solutions to INRIA Dataset (http://pascal.inrialpes.fr/data/human/ identify objects around us by sensors and perform appro- (accessed on December 2020)), which contained 6817 im- priate operations according to movements performed by ages with 3239 pedestrian images, and the image quality in objects. Since in this research we have mainly focused on the this dataset was 227 × 227 pixels. )e results of the re- development of smart transportation in smart cities, so we searchers’ studies showed that the proposed method has a will focus only on identifying pedestrians who are influential high accuracy (approximately 98.8%) for the correct iden- in the development of smart cars and smart transportation, tification of pedestrians compared to other methods of and studies conducted by various researchers. In the field of machine learning. pedestrian identification, we have divided these studies into several groups, examining the studies related to each group Challenge. )e model created on noisy data has not been separately and pointing out the challenges in each. evaluated, and there is a question: if the set of images and input data has a lot of noise, how accurate will the pedestrian 4.1. Studies Conducted in the Field of Pedestrian Identification be identified by this proposed model? in Smart Cities. Belhadi et al. [9] studied the unusual be- Using deep learning, Tome` et al. [11] proposed a system for haviors of pedestrians in smart cities. For this purpose, pedestrian identification. )e researchers also proposed a new several algorithms were proposed, which are basically di- framework for identifying pedestrians. )e researchers also vided into two categories based on performance: proposed new solutions for different stages of pedestrian de- tection, which used deep learning to easily implement their (i) Algorithms that used different data mining and proposed algorithm on modern hardware. To implement and knowledge discovery techniques to discover the evaluate the proposed methods and solutions, they used the relationship between different behaviors of pedes- NVIDIA Jetson TK1, a GPU-based computing platform trians, and finally the knowledge generated to (https://developer.nvidia.com/embedded/jetson-tk1-developer- identify abnormal behaviors of pedestrians kit (accessed on December 2020)), and the Caltech Pedestrian (ii) Algorithms that have been developed based on the dataset (http://www.vision.caltech.edu/Image_Datasets/ history of pedestrian behaviors and based on dif- CaltechPedestrians/ (accessed on December 2020)). )is ferent characteristics of the user to detect abnormal dataset contains about 10 hours of video content related pedestrian behaviors to vehicles collected in different weather conditions. )is To implement these proposed algorithms, the re- dataset had 250 k frames per 137 minutes of video searchers used the HUMBI dataset (https://humbi-data.net/ content with 2300 different pedestrians. Half of the (accessed on December 2020)), which contains 164 attri- frames had no pedestrians, and 30% of the frames had 2 or 3 pedestrians. )e results of the implementation of butes (including gender, age, and physical condition) that include five basic body parts (including face, hands, body, these researchers showed that their proposed method has high efficiency and accuracy in identifying pedestrians in clothes, and eyes), which were designed using the data in this dataset in this study. real time. Journal of Advanced Transportation 7 )is program uses object classification and pedestrian Challenge. To implement these methods, we need large amounts of data, and the more data the dataset uses, the identification and location tracking. )e TensorFlow deep learning framework, Nvidia, cuDNN, and OpenCv accel- more efficient and accurate the proposed method will be. )e challenge arises when data collection for various reasons eration libraries, and the Caltech dataset were used to im- (including privacy) may not be possible in the metropolitan plement, learn, and test the proposed method. )is program areas of many countries in high-traffic urban areas. is installed for deployment in mobile phones or Embedded Systems connected to self-driving cars in order to develop driver assistance systems. 4.2. Studies Conducted in the Field of Pedestrian Identification for the Development of Intelligent Transportation Systems and Challenge. Real-time and accurate detection of objects (such Self-Driving Cars. Chen et al. [12] examined existing ar- as pedestrians) is one of the major challenges in the auto- chitectures for pedestrian detection when using the auto- motive industry to create and develop self-driving cars. With mated driving method. )ese researchers first explained the all the efforts that have been made, the percentage of pe- need to use methods to identify a pedestrian and determine destrian detection accuracy and the speed of detection of his or her route and then discussed the process of identifying existing methods are not enough, so these methods are not a pedestrian while driving a car. )ey, then, discussed how to very acceptable for applying real-time responses. use deep learning techniques (such as R-CNN, SVM) to Ahmed et al. [14] first compared the methods and discover two-step and one-step patterns and test the ef- techniques used to diagnose pedestrians and cyclists. )ey fectiveness of the patterns discovered to identify pedestrians. stated that because of, in the detection stage, the possibility Finally, the researchers examined and compared methods of detecting and locating objects (using deep learning proposed by other researchers to identify pedestrians. )ey techniques such as fast region-convolutional neural network also introduced several datasets (such as KTH, the UCF (R-CNN), faster R-CNN, and single shot detector (SSD)) in series, Hollywood2, and Google AVA) that are used to images and video frames, so the detection stage can be examine proposed methods for detecting pedestrian created as a vital part in creating and developing smart movement. applications in a self-driving vehicle. Finally, tracking results can be used to monitor and identify pedestrians or cyclists. Challenges. In this research, several important challenges in )e main purpose of this study was to investigate the existing identifying pedestrians are mentioned, which are as follows: methods for identifying cyclists. )e results of their studies (i) )e complexity of the environment around the showed that the use of appropriate techniques (e.g., sensor pedestrian can overshadow the operations and fusion and intent estimation) for identifying pedestrians and methods of recognizing the pedestrian and his cyclists can be an important step in maintaining road safety. In this research, first, the challenges in identifying and es- movement and, as a result, make it difficult to ac- curately identify the pedestrian. )erefore, creating timating the purpose and destination are presented, then a methods to identify different perspectives on pe- history of methods proposed by various researchers for destrian detection and operations performed by him pedestrian detection is presented, and the general steps is one of the challenges mentioned in this research. proposed for object detection are explained. Next, the re- search conducted by other researchers on the use of deep (ii) Pedestrian coverage can be extremely effective in the learning techniques and architectures to identify pedestrians process of identifying him/her. If the images are is reviewed, and then, the dataset used by various researchers taken from one perspective, this can affect the ac- to implement their proposed methods for pedestrian and curacy of pedestrian identification and reduce the cyclist detection is explained. accuracy of identification. )erefore, it is necessary for researchers to propose new methods for pre- Challenge. Most of the existing datasets for implementing paring multidimensional images and their simul- object detection techniques are focused on pedestrian de- taneous study and aggregation of the results for tection data, and there is no dedicated dataset to implement early identification of pedestrians, especially in self- the proposed techniques for identifying cyclists, so collecting driving cars. this type of dataset in different areas is currently a challenge. (iii) At present, there is no standard for determining the Zhu et al. [15] studied the challenges of pedestrian operations and actions in a vehicle against various detection using infrared and proposed to use deep movements performed by pedestrians. Better results learning methods for pedestrian detection to overcome can be obtained from the effects of identifying these challenges. By combining deep learning and pedestrians (such as monitoring the safety of pas- background subtraction methods, the researchers pro- sengers and the driver while traveling and managing posed a new method for pedestrian detection. )e pro- environmental pollution in cities) by creating a posed algorithm had two steps for pedestrian detection, classification and stating more details about driving which are as follows: practices in self-driving cars. Said and Barr [13] proposed a new program using deep Step 1: background subtraction methods are performed learning algorithm for fast and accurate pedestrian detection to provide information between frames for the machine to provide real-time responses in driver assistance systems. learning module 8 Journal of Advanced Transportation Challenge. )e dataset used in this study had a very small Step 2: refine Det equipment with a module of attention that is used to improve the accuracy of identifying number of records. )erefore, it seems that a larger dataset can be used to get better results. Also, this method should be pedestrians who are small in stature tested to identify other objects (such as cars and cyclists), In this study, a dataset consisting of infrared videos was and its accuracy should be checked to identify those objects. created that was used to identify pedestrians from a distance Dinakaran et al. [19] proposed generative adversarial and had good performance. networks (GANs) to create a new Cascaded Single Shot Detector (SSD) architecture for remote pedestrian detection. Challenge. )e proposed method in this research is based on In this architecture, DCGAN is used to improve the image a dataset created by the researcher, and its performance is quality for remote pedestrian detection. In this proposed well evaluated. )is is a set of video data stored by infrared. method, several criteria are used to identify the objects in the In order to prove the effectiveness of this method, it seems image. To implement the proposed method, the dataset of that it is necessary to use other datasets that have been the Canadian Institute for Advanced Research (CIFAR) is collected from different geographical locations with different used. )e results obtained from experiments have shown volumes of pedestrian traffic. that the proposed method has a high accuracy in identifying Bunel et al. [16] focused on remote pedestrian detection. vehicles and pedestrians from a distance. When the pedestrian is too far from the camera, the size of Challenge. Generative adversarial networks (GANs) can the pedestrian becomes very small, so it becomes very be used to remotely detect objects in smart cities. Since difficult to detect. )ese researchers suggested a neural security in communications created in IoT-connected net- network-based method and convolutional neural network- works in smart cities is very important and fundamental, so based method to learn the features with an end-to-end we conduct research on the use of GANs in improving approach to identify pedestrians who are too far away from security in smart cities to identify vehicles and pedestrians. the camera and too visible. Further, in this research, to Immediately, it seems very necessary. implement the proposed method, they used Caltech Pe- To overcome the problem of Occlusion handling, Tian destrian Datasets. )e results showed that the proposed et al. [20] proposed DeepParts, which consists of extensive method has a good performance in identifying pedestrians. trackers, instead of using deep learning techniques with an image detector. Some of the features of DeepParts are as Challenge. It seems that the quality of cameras and the follows: amount of pedestrian distance from the camera can be ef- fective in the performance of the proposed method and the (i) First, these DeepParts can be trained with poorly labeled data accuracy of method detection, which has not been con- sidered in this study. )erefore, it is recommended to ac- (ii) Second, DeepParts is able to handle low IoU positive curately define a standard for the best image quality and proposals that shift away from ground truth accuracy of diagnoses with different measurements and (iii) )ird, every part detector in DeepParts is a powerful different photographs or videos. tracker that can detect a pedestrian by observing Haghighat et al. [17] investigated the application of deep only a part of the body learning models in intelligent transportation systems. In the following, the advantages and disadvantages of embedded To implement the proposed method in this research, systems were discussed, and finally, the use of deep learning Caltech and KITTI datasets were used, and its performance techniques to predict the occurrence of traffic on different is compared to other detectors used for pedestrian detection. road routes was examined. Challenge. It is expected that, by using the combination of the results obtained from all the detectors used in the Challenge. All datasets used in research conducted by different parts, the accuracy of identifying objects, especially pedes- researchers have been collected using cameras installed in trians, will be increased. )e use of deep learning techniques different areas. It is expected that, with the advances made in and other techniques on data from detectors may improve the production of self-driving cars and sensors used on cars or pedestrian detection accuracy. on the street floor, the volume of data collected from them will In Navarro Lorente et al. [21], an automated sensor- be greatly increased, so we need new techniques in deep based system was used in applications on self-driving ve- learning to be able to analyze this data. hicles to identify pedestrians. Different types of sensors are Yu et al. [18] proposed a system for tracking and used in self-propelled vehicles, but in this study, researchers identifying pedestrians using deep neural networks, which focused on the Velodyne HDL-64E LIDAR sensor. )e data used a UAV and Kalman Filter forecasting method to track generated by this sensor was analyzed in three dimensions objects and pedestrians, and a dataset (YOLOv3) was used to using machine vision and machine learning algorithms implement the proposed method. To measure the efficiency (such as nearest neighbor algorithm, Bayesian classification of the proposed method for tracking and identifying pe- and support vector machine). A new framework called the destrians, accuracy and execution time and observing and Renault Twizy platform was proposed in this study to de- identifying objects were examined. )e results of experi- velop the ability of self-driving vehicles to identify pedes- mental experiments showed that the proposed method had trians. )e results of the implementation of their proposed fewer errors in identifying pedestrians. framework showed that their selected features along with the Journal of Advanced Transportation 9 cascade classification were used to achieve vehicle detection. algorithms used and the quality of the camera can be im- portant factors in better identifying pedestrians and )e performance evaluation results of the proposed method were about 90% for pedestrians and 88% for vehicle motorcyclists. Challenge. Implementing the algorithms used is time- detection. consuming, and it is necessary to propose methods for accurate and real-time identification of pedestrians and motorcyclists. 4.3. Proposed Methods for Pedestrian Detection Using Dif- Combs et al. [22] focused on using sensors installed on ferent Techniques. Cai et al. [25], to solve the problems self-driving cars to reduce the number of deaths due to caused by resizing objects at the accuracy and speed of object human-caused traffic accidents. )ey used the Fatality identification, provide a deep unified neural network, rep- Analysis Reporting System (FARS) to track the number of resenting the multicast CNN (MS-CNN), for the rapid human error deaths on US urban and suburban roads. )e detection of multifunctional objects. )ey gave MS-CNN researchers hypothesized that a car was traveling on a road including a proposed subnet and an identification subnet. and had all the necessary sensors to detect a pedestrian and )e proposed subnet has several output layers, in which fully effective software to detect and analyze the movements objects are detected at different scales. )e detection subnet and movements made by a pedestrian to identify that pe- uses tracking methods for multipurpose object monitoring. destrian. In addition, sensors mounted on the vehicle itself )e proposed method was implemented on the KITTI and are able to receive signals from the movement of pedestrians. Caltech datasets, and the results showed that the proposed As a result, a model can be developed to be able to easily method has a very good performance in detecting objects identify pedestrians and prevent accidents. )e proposed with a maximum of 15 frames per second. model used data from VLC cameras, radar-based detection systems, and light amplitude detection (LiDAR). )e results Challenge. In this research, the CNN feature approximation of their practical tests showed that, by using these facilities has been used as an alternative to input sampling. )e along with sensors installed on the car body, 90% of acci- challenge is whether other methods can be used to sample dents caused by human error can be prevented, while using the inputs that save more memory and time for calculations. only VLC can reduce the accident statistics by only 30%. Is it possible to increase the speed of moving and replacing Challenge. )ere is high cost of using sensors, VLC. frames (above 15 frames per second)? LiDAR and car-based radar detection systems prevent au- Fukui et al. [26] used complex neural network-based tomakers from using all of them to prevent rising car prices, (CNN) methods that are highly accurate in a variety of or from using low-quality cameras, which could be a reason contexts to identify pedestrians. )e researchers proposed a to reduce its quality and ultimately reduce the accuracy of new method proposed in this research based on CNN and identifying pedestrians or obstacles on the road. used Random Dropout and Ensemble Inference Network Song et al. [23] proposed an algorithm for detecting (EIN) for training and classification, respectively. Random pedestrians on the road. In this study, the pedestrian target Dropout selects units that have a high and variable flexibility area and the results of pedestrian detection on the road by rate for training, while, in a typical dropout, the flexibility combining the algorithm most similar to the neighbor and rate is fixed. EIN creates multiple networks with different the least energy algorithm were accurately divided. In this structures in well-connected layers. )e researchers used the study, all objects that are around a car and can generate Caltech and Daimler Mono pedestrian datasets to imple- traffic for pedestrian identification (such as cyclists, trees, ment their proposed method. other cars, and buildings around cars) are divided. And then, an algorithm was proposed for the environmental coverage Challenge. )e costs of real-time calculations to identify of the road. )e researchers used several experiments to pedestrians using this proposed method are relatively high, evaluate the performance of the traffic-generating object so it is necessary to adopt methods to reduce these costs. classification detection system proposed in this study. )ey To achieve better performance in applying deep learning selected and examined several sequences of images, in- theory to pedestrian detection, Cai et al. [27] improved the cluding different road scenes, different weather conditions, performance of a poorly supervised hierarchical deep and different city streets. learning algorithm with two-dimensional deep belief net- Challenge. )e time required to identify each of the works. In the proposed design of this research, the weak- obstacles (especially pedestrians) is variable, but in changing nesses of the structure and training methods used in the weather conditions and different road conditions, the time various algorithms of the existing classifications are iden- to identify obstacles and pedestrians can be increased or tified, and the following operations are performed to decreased. )erefore, we need to optimize the proposed eliminate these weaknesses: algorithm in this research to realize response time and (i) First, a network of deep one-dimensional beliefs identify pedestrians. expands to two-dimensional, allowing the image Hbaieb et al. [24] proposed a new method for detecting matrix to be loaded directly to preserve more in- the presence of pedestrians in the path of self-driving cars formation from the sample space. through an intervehicle communication system. In this method, descriptor (HOG), support vector machine clas- (ii) Second, a lightweight regulation term is added to sification (SVM), pedestrian tracker, and feature-based performance consistent with the training goal 10 Journal of Advanced Transportation able to identify pedestrians with high accuracy. )is rich without the use of traditional oversight. With this reform, the main training without supervision be- image database can be used in other detectors based on supervised learning architecture. comes weak training under supervision. (iii) )ird, the ability to distinguish between these Challenge. Creating datasets with a number of effective extracted features is created. features for pedestrian detection is one of the important (iv) In this research, the INRIA, Daimler, and CVC challenges that the more datasets we use have a variety of datasets of Spain have been used to implement and effective features for pedestrian detection and can more evaluate the accuracy of the proposed method. accurately identify pedestrians, using the method proposed in this study. Challenge. Working with unstructured data with existing Zeng et al. [33] first focused on in-depth collective public learning about each of the factors used to identify pedes- traditional methods faces several challenges (including challenges in the preprocessing, analysis, and grouping trians using advances in creating a new deep neural network architecture. )e proposed architecture in this research has stages). It is necessary to adopt methods or algorithms to optimize the performance of existing methods for analyzing the following parts: that data and identifying pedestrians through the results of (i) Filtered information maps are obtained from the those analyzes. It is also necessary to adopt strategies to first convolution layer. improve the performance of algorithms and methods used to (ii) From the second convolution layer, maps are ob- classify data and semantic information in occlusion tained to identify parts of the image. conditions. (iii) )e results obtained by identifying each part of the Saeidi and Ahmadi [28] first examined some of the pedestrian body are used to track maps and work DCNN-based learning methods and briefly explained the with information obtained from layers. Argument new algorithms proposed by various researchers for these about access to 20 feature parts or parts of the methods. Next, the researchers proposed a deep architec- pedestrian body is used to estimate the tag (for tural method and a new training method based on parallel example: does a particular window have a pedes- DCNNs for pedestrian detection. )e proposed method had trian or not?). two stages of training, which are as follows: (iv) )e windows are provided in dimensions (height 84 (i) Learning Candidate Pedestrian Extractor Network (CPEN) Candidate for pedestrian training and width 28) that the dimensions of the pedestrian can be identified by 60 by 20. (ii) Parallel training DCNNs (PDCNNs) to teach how to identify a candidate pedestrian by identifying the In other words, the proposed method in this research has body parts of that candidate pedestrian four parts for pedestrian detection, which are feature ex- traction, handling deformation, handling of occasions, and In this study, the Caltech-USA dataset was used to classification. implement the proposed method. )e results obtained from )e proposed method and architecture were imple- evaluating the accuracy of the proposed method in pedes- mented using Caltech and ETH datasets, and their efficiency trian detection and comparing it with other methods showed and accuracy in pedestrian identification were compared that this method has a higher accuracy compared to other with the accuracy of other deep learning methods. )e re- methods. sults show that the accuracy of the proposed method in this research is higher than that of other methods. Challenge. Selecting features for pedestrian detection, es- pecially in multidimensional data, is one of the most im- Challenge. To extract the effective features in high-precision portant challenges when using deep learning techniques. pedestrian detection, we need a large dataset with a large Deep learning techniques (such as SquaresChnFtrs, Infor- number of features that were not available in this study; so, medHaar, and Katamari) performed poorly in selecting to ensure the accuracy of the proposed method in this study, effective features for pedestrian detection, but deep learning we need to prepare a very large dataset with more features. techniques have recently been proposed by various re- Tarchoun et al. [34] proposed two methods for tracking searchers (e.g., CompAct-Deep [29], DeepParts [20], and pedestrians in images taken from moving vehicles: TA-CNN [30]). )ey performed much better in selecting suitable features for pedestrian detection. (i) In the first method, the block matching algorithm Vasconcelos et al. [31] proposed an automated method and block matching features are used to identify for optimizing the efficiency of the training suite by creating pedestrians deformation and creating a local perspective. Using this (ii) )e second method uses a faster R-CNN detector to method, human figures can be identified in the existing detect pedestrians training set by applying monitoring scenarios. Experimental )e proposed methods were implemented using the I2V- results of applying this method to datasets that included a variety of data and images (selection of 16 features from the MVPD database, and the results showed that the first imageNet dataset [32]) showed that if these data were en- method was able to detect pedestrians in images obtained tered as input to a convolutional neural network, it will be from moving vehicles in less time but had a higher false Journal of Advanced Transportation 11 KITTI dataset to implement the proposed method in this positive rate compared to the second method. )e second method had better accuracy and performance in pedestrian research. )e results of the evaluation of the proposed method showed that this method reduces the complexity detection. of the detectors and can be more efficient in accurately Challenge. Neither of these two methods can be used for real- identifying pedestrians. time pedestrian detection applications, so more research is needed to reduce costs and time on these two methods. Challenge. )e proposed method in terms of time required Lee et al. [35] proposed a deep fusion network-based to identify pedestrians may be associated with challenges; pedestrian detection method that used a single shot mul- i.e., in terms of time, more studies should be done on this tibox detector (DSSD) halfway through. )ey use correla- method, so that it can be used immediately to detect pe- tions between other feature maps to create new properties. destrians in cars used. Wagner et al. [38] explored the potential of deep learning In this study, deep fusion network was used to form issues related to the method of recognizing color images at night or techniques in pedestrian identification. )ey examined two deep fusion architectures and their performance on multi- pedestrian images in the dark. KAIST dataset was used to implement the proposed method. )e results obtained from spectral data. Finally, they used a new deep CNN-based the implementation and evaluation of the results showed method to detect pedestrians based on multispectral image that the proposed method, compared to other methods, had data to analyze the proposed method. )ey introduced the at least 4.28% lower error rate in identifying pedestrians in first deep CNN application for pedestrian detection based on the dark environments. multispectral image data, and they used three datasets (including ImageNet [32], CALTECH benchmark [2], and Challenge. Correctly identifying and exacting location of KAIST) to implement and evaluate the proposed method. pedestrians in the dark using existing methods is still a )e evaluation results showed that the proposed method had a higher accuracy in pedestrian detection compared to other challenge. Creating ways to connect different features and deep learning techniques can go a long way in increasing the methods. accuracy of identifying pedestrians in the dark. Ribeiro et al. [36] proposed a deep learning method for Challenge. )e most important challenge in the proposed pedestrian detection (PD) detection in real time to solve method is that, most of the time, early-fusion architecture is problems related to the human-aware robot navigation not able to achieve our expected performance. )e reason for problem. To achieve fast and accurate pedestrian detection this may be due to the inability of the early-fusion network to efficiency, this study developed a combination of Aggregate learn the meaningful multistate abstract properties in a given Channel Features (ACF) detector with a deep convolutional environment. Kim et al. [39] proposed a system with limited re- neural network (CNN). In this method, we have tried to use CNN to increase the accuracy of pedestrian detection by sources for real-world monitoring and identification of moving persons. For this purpose of combination back- trackers. To implement the proposed method and evaluate its accuracy, two sets (called corridor and Mbot) were used, ground subtraction and convolutional neural networks (CNNs), they used it to identify and detect moving objects which have real photos taken by the cameras (photos col- lected from the cameras in the internal and external sensors using outdoor CCTV videos. )e background subtraction of the robot), and a typical robot navigation environment algorithm used to find the desired areas in the video frame was used to evaluate the accuracy of the method in iden- and the CNN classifier was used to classify the ROIs tifying pedestrians, and the results showed that it has suf- obtained in one of the predefined classes. To implement ficient speed and accuracy to be used in these environments the proposed method in practice, various datasets collected and robot navigation applications to identify pedestrians. by several real-world CCTV cameras were used. )e re- sults showed that the proposed system had a high accuracy Challenge. )e performance of the proposed method should in identifying pedestrians and was also less complex than other methods. be evaluated on datasets collected from cameras located in different places with different light intensities and distances, Challenge. Occurrence of some problems in the collected different types of sensors installed in the environment such as laser sensors. data can reduce the performance or accuracy of pedestrian Hu et al. [37] worked to create a powerful pedestrian detection. For example, lack of training data may disrupt the detector. For this purpose, the researchers designed the training process. On the other hand, using the same images deep convolutional neural network (DCNN) as an image will cause a pedestrian to be repeatedly identified several feature to teach a set of enhanced decision models, using times, and this will reduce the performance of the proposed redesigned learning algorithms (CFMs) without the use of system for pedestrian detection. Lin et al. [40] proposed a framework for pedestrian learning algorithms. To increase the efficiency and accu- racy of DCNN-based detectors for image detection of detection that is based on incorporating pixel-wise infor- mation into deep convolutional feature maps. In this con- pedestrians, hand-crafted features such as optical flow are used. In this study, they reviewed various datasets that text, they used the zooming properties to improve image have been used by other researchers to implement their quality to help easily and accurately identify pedestrians. proposed methods for pedestrian detection. )ey used the )erefore, the proposed method in this research helps 12 Journal of Advanced Transportation Table 1: Challenges and proposed solutions. Challenges Proposed solutions Installation of devices in the desired routes to monitor the passage of Lack of structured data in all studied routes to identify pedestrians pedestrians and bicycles around the clock and the use of various data at different times of the day with different light intensities during mining techniques and deep learning to create patterns to identify the day and different weather conditions cyclists and pedestrians in different weather conditions and different light intensities Absence of any known standard for selecting appropriate and essential features from the features collected for pedestrian Using more advanced sensors in different directions or applying identification or limited features used to model pedestrian sensors in the car body, creating a centralized database to consolidate behaviors and the need to apply metaheuristic algorithms to solve data collected from sources and sensors used in different places complex intelligent computing problems Creating high-precision and high-quality sensors and cameras, Existence of noise in various images and data collected, poor quality installing high-quality cameras on sensitive routes, proposing new or very low quality of some images collected by cameras installed on techniques for preprocessing data and images, and accurately the road, especially in cloudy, rainy and icy weather or dark at night detecting noise images Establish protocols and standards for collecting data on public places, Lack of access to clear and uninterrupted data and images for establish agreements and laws to protect the privacy of the public research due to privacy while collecting 24-hour data from all road routes )e complexity of the environment around the pedestrian can overshadow the operations and methods of identifying the Develop methods to identify different perspectives on pedestrian pedestrian and her movement, and as a result, make it difficult to detection and operations performed by his/her accurately identify the pedestrian Pedestrian coverage can be very effective in the process of Proposing new methods by researchers to prepare multidimensional identifying him. If the images are taken from one perspective, it can images and their simultaneous study and aggregation of the results affect the accuracy of pedestrian identification and reduce the for early identification of pedestrians, especially in cars accuracy of identification Real-time and accurate detection of objects (such as pedestrians) is Propose methods to increase the accuracy of pedestrian detection and an important challenge for car companies to create and develop speed up the process of immediate response to avoid accidents when self-driving cars detecting obstacles and pedestrians Collect data on cyclists on different routes with different light Lack of specific data to diagnose cyclists intensities and create patterns to identify cyclists )e quality of the cameras, the amount of pedestrian distance from the camera can affect the performance of the proposed method and Define a standard for the best image quality and detection accuracy the accuracy of the method detection, which has not been with different measurements and different shots or videos considered in many studies )e cost of performing real-time calculations to identify It is necessary to adopt methods to reduce these costs pedestrians using this proposed method is relatively high It is necessary to adopt methods or algorithms to optimize the performance of existing methods for analyzing that data and Working with unstructured data with existing traditional methods identifying pedestrians through the results of those analyzes. It is also faces several challenges (including the challenges of preprocessing, necessary to adopt strategies to improve the performance of analysis, and grouping) algorithms and methods used to classify data and semantic information in occlusion conditions identify pedestrians who are seen in a very small image by the performance of these detectors using various datasets inserting geographical location specifications and pedestrian including Caltech. In this study, the most widely used features. )e proposed method uses three datasets: Caltech datasets were briefly described, and the strengths and [41], INRIA [42], and KITTI [43]. )e implementation weaknesses of each were expressed. )ree features (in- results obtained from the evaluation and comparison with cluding best features, additional data, and background/ other methods showed that this method is more efficient in conceptual information) were used to conduct practical terms of reducing the time of pedestrian identification and experiments in this study, which can affect the efficiency of the proposed method for pedestrian detection. )ree im- the number of unidentified cases. portant and famous trackers (including deformable part Challenge. Due to the small size of the pedestrian image, it models, decision forests, and deep networks) are based on seems that there are complications in recognizing of that the different learning techniques used. pedestrian in an image taken from a low light environment, especially at night, using the method proposed in this research. Challenge. It seems that the most important challenge in the Dollar ´ et al. [41] reviewed advances over the past decade field of pedestrian detection is to develop a deeper under- in developing methods for pedestrian detection and pro- standing in selecting the best features to achieve the highest posing 40 trackers for pedestrian detection. )ey analyzed accuracy and performance in real-time pedestrian detection. Journal of Advanced Transportation 13 accessible surface area of proteins by iterative deep learning,” )e main challenge ahead seems to develop a deeper un- Scientific Reports, vol. 5, no. 1, p. 11476, 2015. derstanding of what makes good features, so as to enable the [8] S. Dargan and M. Kumar, “A survey of deep learning and its design of even better ones. applications: a new paradigm to machine learning,” Archives of Computational Methods in Engineering, vol. 27, pp. 1071– 5. Conclusion 1092, 2019. [9] A. Belhadi, Y. Djenouri, G. Srivastava, D. Djenouri, Nowadays, the amount of data generated during a day from J. C.-W. Lin, and G. Fortino, “Deep learning for pedestrian various sensors and other devices is enormous, and collective behavior analysis in smart cities: a model of group technologies such as cloud computing are being used to trajectory outlier detection,” Information Fusion, vol. 65, help store large volumes of collected data. One of the pp. 13–20, 2021. benefits of analyzing this data is the discovery of knowledge [10] B. Kim, N. Yuvaraj, K. R. Sri Preethaa, R. Santhosh, and and a pattern for use in similar situations by training A. Sabari, “Enhanced pedestrian detection using optimized deep convolution neural network for smart building sur- machines. In this study, we explained deep learning and its veillance,” Soft Computing, vol. 24, no. 22, pp. 17081–17092, difference with machine learning, and then some research is done by various researchers around the use of deep [11] D. Tome, ` F. Monti, L. Baroffio, L. Bondi, M. Tagliasacchi, and learning techniques in the creation and development of S. Tubaro, “Deep convolutional neural networks for pedes- smart cities and pedestrian identification in smart cities trian detection,” Signal Processing: Image Communication, and Intelligent Transportation Systems (ITS). Finally, we vol. 47, pp. 482–489, 2016. examined smart transportation and listed the challenges in [12] L. Chen, N. Ma, P. Wang et al., “Survey of pedestrian action each of them. recognition techniques for autonomous driving,” Tsinghua In general, according to the studies conducted in this Science and Technology, vol. 25, no. 4, pp. 458–470, 2020. paper, the most important challenges in identifying pe- [13] Y. F. Said and M. Barr, “Pedestrian detection for advanced destrians on the street using the proposed technologies and driver assistance systems using deep learning algorithms,” methods, especially deep learning techniques, can be IJCSNS (International Journal of Computer Science and Network Security), vol. 19, no. 10, 2019. expressed in Table 1 and some of the solutions that seem [14] S. Ahmed, M. N. Huda, S. Rajbhandari, C. Saha, M. Elshaw, useful in solving these challenges are suggested in this table. and S. Kanarachos, “Pedestrian and cyclist detection and intent estimation for autonomous vehicles: a survey,” Applied Data Availability Sciences, vol. 9, no. 11, p. 2335, 2019. [15] Y. Zhu, J. Yang, X. Xieg, Z. Wang, and X. Deng, “Long- )is is a survey, and we introduced already all databases in distanceinfrared video pedestrian detection using deep this study. learning and backgroundsubtraction,” Journal of Physics: Conference Series, vol. 1682, 2020. [16] R. Bunel, F. Davoine, and P. Xu, “Detection of pedestrians at Conflicts of Interest far distance,” in Proceedings of the 2016 IEEE International )e authors declare that they have no conflicts of interest. Conference on Robotics and Automation (ICRA), IEEE, Stockholm, Sweden, May 2016. [17] A. K. Haghighat, V. Ravichandra-Mouli, P. Chakraborty, References Y. Esfandiari, S. Arabi, and A. Sharma, “Applications of deep learning in intelligent transportation systems,” Journal of Big [1] R. E. Schapire and Y. Freund, Foundations of Machine Data Analytics in Transportation, vol. 2, no. 2, pp. 115–145, Learning, MIT Press, Cambridge, MA, USA, 2012. [2] ETSI, “Intelligent transport systems (ITS); security; threat, [18] H. Yu, G. Li, W. Zhang et al., “)e unmanned aerial vehicle vulnerability and risk analysis (TVRA),” European Tele- benchmark: object detection, tracking and baseline,” Inter- communications Standards, Sophia Antipolis, France, ETSI national Journal of Computer Vision, vol. 128, no. 5, TR 102 893, 2010. pp. 1141–1159, 2020. [3] T. Rawal and V. Devadas, “Intelligent transportation system [19] R. K. Dinakaran, P. Easom-McCaldin, A. Bouridane et al., in India—a review,” Journal of Management Development, “Deep learning based pedestrian detection at distance in smart vol. 2, p. 299, 2015. cities,” in Intelligent Systems and Applications, Springer, [4] P. Fernandes and U. Nunes, “Vehicle communications: a Cham, Germany, 2019. short survey,” in Proceedings of the IADIS Telecommunica- [20] Y. Tian, P. Luo, X. Wang, and X. Tang, “Deep learning strong tions, Networks and Systems, p. 1556, Lisboa, Portugal, July parts for pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, [5] K. B. Smith, “Typologies, taxonomies, and the benefits of policy classification,” Policy Studies Journal, vol. 30, no. 3, December 2015. [21] P. J. Navarro Lorente, C. Fernandez, R. Borraz, and D. Alonso, pp. 379–395, 2002. [6] T. Brosch, R. Tam, and Initiative for the Alzheimers Disease “A machine learning approach to pedestrian detection for autonomous vehicles using high-definition 3D range data,” Neuroimaging, “Manifold learning of brain MRIs by deep learning,” in Proceedings of the International Conference on Sensors, vol. 17, no. 1, p. 18, 2017. [22] T. S. Combs, L. S. Sandt, M. P. Clamann, and N. C. McDonald, Medical Image Computing and Computer-Assisted Intervention, Nagoya, Japan, September 2013. “Automated vehicles and pedestrian safety: exploring the promise and limits of pedestrian detection,” American Journal [7] R. Heffernan, K. Paliwal, J. Lyons et al., “Improving prediction of secondary structure, local backbone angles, and solvent of Preventive Medicine, vol. 56, no. 1, pp. 1–7, 2019. 14 Journal of Advanced Transportation [23] Y. Song, J. Yao, Y. Ju, Y. Jiang, and K. Du, “Automatic de- [37] Q. Hu, P. Wang, C. Shen, A. v. d. Hengel, and F. Porikli, tection and classification of road, car, and pedestrian using “Pushing the limits of deep CNNs for pedestrian detection,” IEEE Transactions on Circuits and Systems for Video Tech- binocular cameras in traffic scenes with a common frame- nology, vol. 28, no. 6, pp. 1358–1368, 2017. work,” Complexity, vol. 2020, Article ID 2435793, 17 pages, [38] J. Wagner, V. Fischer, M. Herman, and S. Behnke, “Multi- spectral pedestrian detection using deep fusion convolutional [24] A. Hbaieb, J. Rezgui, and L. Chaari, “Pedestrian detection neural networks,” in Proceedings of the 24th European Sym- for autonomous driving within cooperative communica- posium on Artificial Neural Networks, Computational Intel- tion system,” in Proceedings of the 2019 IEEE Wireless ligence and Machine Learning (ESANN), Bruges, Belgium, Communications and Networking Conference (WCNC), April 2016. IEEE, Marrakesh, Morocco, April 2019. [39] C. Kim, J. Lee, T. Kan, and Y.-M. Kim, “A hybrid framework [25] Z. Cai, Q. Fan, R. S. Feris, and N. Vasconcelos, “A unified multi- combining background subtraction and deep neural networks scale deep convolutional neural network for fast object detec- for rapid person detection,” Journal of Big Data, vol. 5, no. 1, tion,” in European Conference on Computer Vision, Springer, pp. 1–24, 2018. Cham, Germany, 2016. [40] C. Lin, J. Lu, G. Wang, and J. Zhou, “Graininess-aware deep [26] H. Fukui, T. Yamashita, Y. Yamauchi, H. Fujiyoshi, and feature learning for pedestrian detection,” in European H. Murase, “Pedestrian detection based on deep convolu- Conference on Computer Vision (ECCV), Springer, Cham, tional neural network with ensemble inference network,” in Germany, 2018. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium [41] P. Dollar, ´ C. Wojek, B. Schiele, and P. Perona, “Pedestrian (IV), IEEE, Seoul, South Korea, July 2015. detection: a benchmark,” in Proceedings of the 2009 IEEE [27] Y. Cai, Y. He, H. Wang, X. Sun, L. Chen, and H. Jiang, Conference on Computer Vision and Pattern Recognition, June “Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model,” International [42] N. Dalal and B. Triggs, “Histograms of oriented gradients for Journal of Advanced Robotic Systems, vol. 14, no. 1, Article ID human detection,” in Proceedings of the 2005 IEEE Computer 1729881417692311, 2016. Society Conference on Computer Vision and Pattern Recog- [28] M. Saeidi and A. Ahmadi, “Deep learning based on parallel nition (CVPR’05), IEEE, San Diego, CA, USA, June 2005. CNNs for pedestrian detection,” International Journal of [43] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for au- Information and Communication Technology Research, vol. 10, tonomous driving? the kitti vision benchmark suite,” in no. 4, pp. 42–52, 2018. Proceedings of the 2012 IEEE Conference on Computer Vision [29] Z. Cai, M. Saberian, and N. Vasconcelos, “Learning com- and Pattern Recognition, IEEE, Providence, RI, USA, June plexity-aware cascades for deep pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, December 2015. [30] Y. Tian, P. Luo, X. Wang, and X. Tang, “Pedestrian detection aided by deep learning semantic tasks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015. [31] C. N. Vasconcelos, A. Paes, and A. Montenegro, “Towards deep learning invariant pedestrian detection by data en- richment,” in Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, Anaheim, CA, USA, December 2016. [32] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, “Imagenet: a large-scale hierarchical image database,” in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, June 2009. [33] X. Zeng, W. Ouyang, and X. Wang, “Multi-stage contextual deep learning for pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, December 2013. [34] B. Tarchoun, A. Khalifa, S. Dhifallah, I. Jegham, and M. Mahjou, “Hand-crafted features vs deep learning for pedestrian detection in moving camera,” Traitement du Sig- nal, vol. 37, no. 2, pp. 209–216, 2020. [35] Y. Lee, T. D. Bui, and J. Shin, “Pedestrian detection based on deep fusion network using feature correlation,” in Proceedings of the 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, Honolulu, HI, USA, November 2018. [36] D. Ribeiro, R. Mateus, P. Miraldo, and J. C. Nascimento, “A real-time deep learning pedestrian detector for robot navi- gation,” in Proceedings of the 2017 IEEE International Con- ference on Autonomous Robot Systems and Competitions (ICARSC), IEEE, Coimbra, Portugal, April 2017. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Advanced Transportation Hindawi Publishing Corporation

Applications of Deep Learning Techniques for Pedestrian Detection in Smart Environments: A Comprehensive Study

Loading next page...
 
/lp/hindawi-publishing-corporation/applications-of-deep-learning-techniques-for-pedestrian-detection-in-WyuqLA0W0q

References (43)

Publisher
Hindawi Publishing Corporation
Copyright
Copyright © 2021 Fen He et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
ISSN
0197-6729
eISSN
2042-3195
DOI
10.1155/2021/5549111
Publisher site
See Article on Publisher Site

Abstract

Hindawi Journal of Advanced Transportation Volume 2021, Article ID 5549111, 14 pages https://doi.org/10.1155/2021/5549111 Review Article Applications of Deep Learning Techniques for Pedestrian Detection in Smart Environments: A Comprehensive Study 1 2 3 4 Fen He , Paria Karami Olia , Rozita Jamili Oskouei , Morteza Hosseini , 5 2 Zhihao Peng , and Touraj BaniRostam School of Information Engineering, Guangzhou Nanyang Polytechnic College, Guangzhou, Guangdong, China Computer Engineering Department, Islamic Azad University, Central Tehran Branch, Tehran, Iran Department of Computer Science and Information Technology, Mahdishahr Branch, Islamic Azad University, Mahdishahr, Iran Department of Engineering, Islamic Azad University, South Tehran Branch, Tehran, Iran School of Software, Dalian Neusoft University of Information, Dalian, China Correspondence should be addressed to Rozita Jamili Oskouei; rozita2010r@gmail.com Received 8 February 2021; Revised 1 April 2021; Accepted 24 August 2021; Published 4 October 2021 Academic Editor: Chunjia Han Copyright © 2021 Fen He et al. )is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Intelligent transportation systems have been very well received by car companies, people, and governments around the world. )e main challenge in the world of smart and self-driving cars is to identify obstacles, especially pedestrians, and take action to prevent collisions with them. Many studies in this field have been done by various researchers, but there are still many errors in the accurate detection of pedestrians in self-made cars made by different car companies, so in the research in this study, we focused on the use of deep learning techniques to identify pedestrians for the development of intelligent transportation systems and self- driving cars and pedestrian identification in smart cities, and then some of the most common deep learning techniques used by various researchers were reviewed. Finally, in this research, the challenges in each field are discovered, which can be very useful for students who are looking for an idea to do their dissertations and research in the field of smart transportation and smart cities. driver is drowsy or the driver does not pay attention to 1. Introduction hazards to prevent accidents. In recent years, intelligent transportation systems have been Some of the most important problems in creating and developed to help reduce the volume of traffic in metro- developing self-driving cars are as follows: politan areas, reduce the rate of accidents and injuries and (i) Lack of accurate patterns to identify pedestrians deaths caused by them, reduce fuel consumption, reduce and roadblocks with very high accuracy in different environmental pollution, and so on. )ese systems use roads with different light intensities and different different technologies (including IoT, machine learning and image quality data mining, neural networks, deep learning, and image processing) for various applications. On the other hand, (ii) Error in identifying pedestrians and obstacles in the paths of self-driving cars large automotive and technology companies (such as Google and Tesla) are trying to produce self-driving smart cars that (iii) People’s distrust of self-driving cars can provide safe travel for people when the driver is drowsy (iv) Lack of acceptance of people for 24-hour control and that can save his life as well as passengers’ lives, with and monitoring of various urban and interurban automatic control car to prevent any accident. )ese vehicles roads’ paths must be equipped with sensors to sense the environment and (v) Negative effects of dark weather and snow/ice and identify objects close to the car and inform the driver and also have actuators to perform real-time operations when the fog and rain on the quality and performance of 2 Journal of Advanced Transportation cameras installed on cars, ultimately reducing the In recent years, organizations responsible for transport accuracy of pedestrian detection and obstacles in management in different countries of the world have shown these vehicles great attention to the development and use of intelligent transport systems using the creation of intercity networks. (vi) Lack of necessary infrastructure to implement in- )e main reason for this is the diverse applications of au- telligent transportation on all urban and interurban tomotive communication technology in the four areas of roads safety promotion, mobility improvement, environmental (vii) )e need for high investment to implement the protection, and asset management. Automotive intelligent necessary infrastructure to implement intelligent communication systems provide technical and economic transportation and communication systems to solutions to the transportation challenges of the 21st century. connect vehicles to each other (V2V) and vehicles )ese systems must be able to enable a growing segment of with roadside infrastructure (V2I) the human population to move freely, without the risk of Machine learning (ML) is a field of artificial intelligence that accidents and with minimal fuel consumption and envi- uses statistical techniques to learn hidden patterns from existing ronmental pollution [2]. data and to make decisions about unseen records. )e main task Intelligent transportation technology ranges from basic management systems (such as car navigation, traffic signal of a machine learner is building a general model on the possible distribution of training examples and then generalizing expe- control systems, variable driving signs, automatic license plate number recognition, or speed camera) to surveillance appli- rience to unseen examples [1]. )e learning process depends on the quality of the data displayed. An example is presented in a cations such as advanced CCTV security systems that provide information. From sources such as car park guide information dataset with different properties. Unfortunately, extracting ef- ficient features can be difficult for some tasks. systems, they collect weather information, etc. [2]. Deep learning is an advanced branch of the ML disci- Intelligent Transportation System (ITS) means the use of pline that aims to discover complex representations of a set of tools, facilities, and expertise such as traffic engi- simpler representations. Deep learning methods are usually neering concepts, software, hardware, and telecommuni- cations technologies in a coordinated and integrated manner based on artificial neural networks consisting of several hidden layers with nonlinear processing units. )e word to improve the efficiency and security of transportation systems. deep refers to several hidden layers used to change the display of data. Using the concept of feature learning, each Intelligent transportation systems can also be general- ized to different modes of transportation, in which, using hidden layer of neural networks plots its input data in a new display. )e layer manages to absorb a higher level of ab- automated tools and related scheduling, various types of straction than the abstract concept in the previous layer. In information receiving and processing operations, as well as deep learning architectures, the hierarchy of features learned traffic management and control, are performed. In this at multiple levels is finally mapped to the output of the ML system, by limiting the role of human factors in information work in a single framework. Similar to ML methods, the processing or control and management processes, we im- deep learning architecture is divided into two broad cate- prove the quality of decision-making and management processes. gories: (a) unsupervised learning methods and (b) super- vised learning approaches, including deep neural networks. In intelligent transportation systems (ITS), the definition of transportation infrastructure in addition to information )is research is organized in five sections. In the second part, we provide a complete description of intelligent and communication technologies leads to the achievement of goals such as improving passenger safety, reducing transportation systems. In the third section, we provide a brief overview of deep learning and some of its applications. transportation time, and reducing fuel consumption and In the fourth section, we describe the use of deep learning to wear or tear of car tires. Applications of ITS include accident identify pedestrians in smart cities and intelligent trans- management, electronic toll collection management, public portation systems, review some of the research conducted by transportation management, passenger communication various researchers in this field, and state the challenges in management, and traffic flow management [3]. each area of research. Finally, in the fifth section, we will Compared to traditional traffic engineering, the intelli- gent transportation system (ITS) has created a new trans- provide the conclusion. portation system. Due to different national circumstances, the development priorities of these systems are different, and 2. Intelligent Transport Systems (ITS) therefore, the content of ITS research is not the same in all countries. In general, ITS uses information, communication, Intelligent transportation systems, for automatic road man- agement and real-time operations in the event of an natural control, computer technology, and other current technol- accident (such as a mountain fall, avalanche, and icy road ogies to create a real-time, accurate, and efficient trans- floor) or unnatural disasters (such as accidents, road repairs, portation management system. and car traffic), have been developed. )ese systems use sensors as a tool to identify and understand the state of the travel environment. )e data collected by these sensors using 2.1. Architecture of Intelligent Transportation Systems. )e US Department of Transportation, through RITA Research communication technologies (such as WiFi and DSRC) to other vehicles on the route are sent to control centers. and Innovation Technology Management, defined a Journal of Advanced Transportation 3 2.2. Some Services Provided in Intelligent Transportation national architecture for ITS and provided a common structure for designing intelligent transportation systems. Systems. Some of the most common services provided in intelligent transportation systems are briefly described in )e ITS function model (Logic Architecture) provides a functional view of ITS user services. Physical architecture this section [5]. divides the functions defined by logical architecture into classes and subsystems. Figure 1 shows the high-level 2.2.1. Accident Management. We divide the stages of acci- diagram of the proposed physical architecture (Archi- dent management into five stages (depending on the type tecture Development Team 2007a), in which 22 subsystems and severity of the accident) in which one or more stages (white rectangles) are distributed among four classes: may occur simultaneously [5]: passengers, centers, vehicles, and the field or area of (i) Detection and notification: it is the detection and operation. notification of accidents that are often used in In Figure 1, the communication requirements between mobile phones at this stage. these subsystems are supported by four types of commu- nication, which are shown in the form of an oval in the (ii) Verification: the existence of the accident and the figure: wireless communication over a wide area, fixed exact type and location of the accident (via traffic point-to-fixed point communication, vehicle-to-vehicle surveillance cameras) are confirmed. communication, and dedicated short-range communica- (iii) Incident site management: it is a complex process tions [4]. that requires careful coordination, communication, )e following is a brief description of each of the classes and cooperation between the people present on the in Figure 1: scene, all supporting institutions and the general (i) Travelers: different services are provided to pas- public. Important points for proper incident scene management include the following: sengers (including drivers and occupants of cars), which are generally grouped into two categories, (i) Providing accurate information to the dispatch which are as follows [4]: unit in a timely manner, including the exact (i) Support to their remote travel: using the in- location of the accident, the severity of the stallation of surveillance cameras by the Road accident, and so on (ii) Establishing evidence in a safe area to minimize and Transportation Administration on various routes inside and outside the city, road and oncoming traffic risk given the location of damaged vehicles and rescue personnel transport managers can monitor the move- ments of passengers on different routes from the (iii) Establishing a command system, especially headquarters and in case of any problem, when major events occur whether it is an accident or a fall of a mountain, (iv) Asking for help from cleaning companies if and etc., and send a team to the place there is a possibility of hazardous substances at immediately the scene (ii) Access to personal information: it includes (v) Public information in case of an accident monitoring of the crossing of intersections through mass media, Internet, or SMS to when the traffic lights are red and registering the passengers license plate number of the offending vehicle (iv) Protection of evidence: it includes the protection of and then issuing a fine for him evidence at the scene of the accident and potential (ii) Control centers including the centers providing evidence that may later be used to prosecute the perpetrator or analyze the data in the future. Any necessary and useful information for drivers, traffic management centers, relief and emergency centers, suspicious items at the scene (such as guns, bullets, drugs, and alcohol) should also be protected to be transport management centers, toll collection cen- ter, collected data management centers, transport handed over to the police on the scene or the highway warden [5]. fleet control centers, and road maintenance management (v) Hazardous Materials: when hazardous materials (iii) Vehicles including personal vehicles, emergency are spilled at the scene of an accident, they must vehicles, commercial vehicles, freight vehicles, first be thoroughly inspected by police dis- support vehicles and relief vehicles, or vehicles patchers, and then, the necessary measures must belonging to the police patrol be taken to collect those materials and clean the environment. (iv) Roadside equipment including equipment installed (vi) Breakdown and Demobilization: public mobiliza- on the road, equipment related to toll collection, parking management, checking commercial vehi- tion to clear the scene and analyze the accident that occurs when all injured people, damaged vehicles, cles, and checking the weight of trucks with their load, which, if it is more than a certain weight, equipment, and debris are removed from the scene. should have stopped and in addition to fining them, Public mobilization is to create security, expedient, their burden should also be reduced and regular departure of all those present at the 4 Journal of Advanced Transportation Advanced Traveler ATIS Information Systems ✓ traffic information ✓ travel guidance Advanced Traffic ATMS ✓ optimized route guidance Management System ✓ Rcal-time traffic control ITS Advanced Public ✓ Incident management Intelligent Transportation Systems ✓ Automatic vehicle APTS Transport enforcement System ✓ Public tranportation ✓ Automatic Toll collection information ✓ Public traffic management Advanced Vehicle AVHS Highway System Commercial Vehicle ✓ AVS:Advanced Vehicle System CVO Operations - Vehicle Automatic Control System ✓ FFM: Freight and Fleet Management ✓ AHS: Advanced Highway System ✓ HMM: Hazard Material Management Figure 1: High-level architecture diagram for ITS [4]. scene of the accident and equipment and vehicles systems, and vehicle connectors) are combined to form a from the scene and return the affected area to cohesive interface that is capable of parsing and has real-time data analysis and decision-making about the current traffic normal with normal traffic flow [5]. situation and the assessment of subsequent conditions that may occur, as well as the adoption of appropriate measures 2.2.2. APTS Public Transportation Management. )ese to deal with the conditions that have arisen: dynamic traffic systems use new information management technologies to control systems, highway operations management systems, increase the efficiency and enhance the security of public accident prevention systems, and making necessary and transportation systems. )ese systems include instantaneous appropriate decisions when accidents occur, etc. )ey are and real-time passenger information management systems, considered as advanced traffic management systems [5]. vehicle location detection systems, bus arrival time notifi- cation systems, and bus crossing priority prioritization 2.2.5. Network Security Management. )e main purpose of systems. network safety management, somewhat like the manage- ment of sensitive points, means identifying the areas where 2.2.3. Advanced ATIS Passenger Information Systems. accidents are most likely to occur. )erefore, there is an )ese systems provide information on travel routes and urgent need to ensure road safety in those areas. However, weather conditions for transportation system users, so that there are two important differences between hotspot man- they can make the right decisions to choose the route, es- agement and network security management [5]: timate travel time, and avoid getting caught in crowded (i) In network safety management, the important goal is routes. Several technologies are used for this purpose, which to identify roads with different degrees of security and are as follows [5]: ultimately to identify accident hotspots or sensitive (i) GPS enabled in car navigation systems points in the road system (such as intersections). (ii) Dynamic signs and messages for timely and real- (ii) In network safety management, a report on the severity time notification in traffic, turns and passes, and of accidents is prepared, and accident-prone parts of accidents or when the road is closed for various the road are identified. In the management of critical or reasons such as repairs accident-prone points, the number of accidents at each critical point is usually too high, so this point is given (iii) Websites are used to indicate congestion on high- more importance than the severity of the accident. ways, main streets, and urban and interurban road networks 3. Deep Learning 2.2.4. Advanced ATMS Traffic Management Systems. )e Learning is the process by which a system improves its data and information obtained through different subsystems performance by using past experiences. Since 2006, deep (such as vehicle type identifiers, in-vehicle messaging learning has emerged as a new subfield of machine learning, Journal of Advanced Transportation 5 (i) )e training phase involves labeling large amounts affecting a wide range of signal and information processing in both traditional and modern fields. Many traditional of data and determining their adaptive properties. machine learning and signal processing techniques use (ii) )e inference step is to conclude and label new and special architectures that contain a single layer of nonlinear unseen data, using their prior knowledge. Deep features. learning is a method that helps the system under- Some examples of deep learning in the workplace in- stand the complex tasks of perception with maxi- clude a self-propelled vehicle slowing down as it approaches mum accuracy. Deep learning is also known as deep a pedestrian crossing, an ATM rejecting a counterfeit structured learning and is a hierarchical learning that banknote, and a smartphone app instantly translating an consists of several layers that include nonlinear installed signboard performing on the street. Deep learning processing units to convert and extract features. is especially suitable for identification programs such as face Each subsequent layer takes the results from the recognition, text translation, voice recognition, and ad- previous layer as input. vanced driver assistance systems, including and symptom )e learning process is performed using the distinct recognition [6]. stages of abstraction and multiple levels of representation in a supervised or unsupervised manner. Deep learning or deep neural network uses a basic computing unit, a neuron that 3.1. .e Difference between Deep Learning and Machine receives multiple signals as input. It integrates these signals Learning. Deep learning is one of the subfields of machine linearly with the weight and transmits the combined signals learning. By learning the machine, the features of an image to the nonlinear tasks to produce output. can be extracted manually. With deep learning, raw images In the “deep learning” method, the term “deep” refers to can be inserted directly into a deep neural network that the multiple layers through which data is converted. )ese learns features automatically. Deep learning usually requires systems are composed of a very special deep credit allo- hundreds of thousands or millions of images to get the best cation (CAP) path, which means that the steps were per- results, while machine learning works well with small formed to convert the input to output and represent the datasets. Deep learning is also computationally intensive and impact connection between the input layer and the output requires a high-performance CPU [7]. layer [7]. It should be noted that there is a difference be- Deep learning is the most effective, supervised, and cost- tween deep learning and machine learning. Machine effective machine learning approach. Deep learning is not a learning involves a set of methods that help the machine limited learning method, but it follows a variety of methods receive raw data as input and set views for the purpose of and topographies that can be used to make broad predictions detection and classification. Deep learning techniques are about complex problems. )is technique includes descrip- simply a type of learning method that has several levels of tive and distinctive features in a completely categorized way. representation and is at a more abstract level. Figure 2 Deep learning methods with remarkable performance have shows the difference between machine learning and deep achieved significant success in a wide range of applications learning. with useful security tools. Deep learning is used in many Deep learning techniques in large databases use non- applications, including business, comparative experiments, linear transformations and high-level model abstraction. biological image classification, computer insight, cancer )ey also describe how a machine can change the internal detection, natural language processing, object recognition, features needed to count descriptions in each layer by face recognition, handwriting, speech recognition, stock accepting abstractions and displaying previous layers. )is market analysis, and creation and the development of smart new learning approach is widely used in the areas of adaptive cities. testing, big data, cancer diagnosis, data flow, document Machine learning is a subset of artificial intelligence (AI) analysis and identification, healthcare, object recognition, that gives systems the benefits of automatically learning speech recognition, image classification, pedestrian detec- concepts and knowledge without explicit planning. It begins tion, natural language processing, and voice activity with observations such as direct experiences to prepare detection. features and patterns in the data and to produce better )e deep learning model uses a set of features set for results and decisions in the future. Deep learning relies on a large features using bulk dataset for unique features, then set of machine learning algorithms that model high-level extracts a classification model, and creates an integrated abstractions in data with multiple nonlinear transforma- classification to explore a variety of applications. tions. Deep learning technology works on an artificial neural )e key factors on which the deep learning method is network (ANN) system. )ese neural networks continuously based are as follows [8]: use learning algorithms, and by constantly increasing the amount of data, the efficiency of training processes can be (i) Nonlinear processing in multiple layers or stages: nonlinear processing in multiple layers refers to a improved. )e efficiency of deep learning algorithms de- pends on the volume of large data. )e process is called deep hierarchical method in which the present layer ac- cepts the results of the previous layer and transmits training, because the number of neural network levels in- creases over time. its output as input to the next layer. Hierarchy is )e operation in the deep learning process generally created between layers to organize the importance of the data. depends on two stages called the training and inference. 6 Journal of Advanced Transportation Machine Learning )e results of this study showed that the use of deep learning techniques in comparison with the use of data Car mining techniques both reduces the time of analysis and Not Car detection of normal and abnormal behaviors and in- Input Feature extraction Output Classification creases the accuracy of identifying abnormal behaviors of pedestrians. )e researchers pointed out that modeling Deep Learning pedestrian behaviors and behavioral analysis in the de- velopment of smart cities can help increase the efficiency Car of smart agents used for various applications in these Not Car cities. Input Feature extraction • Classification Output Figure 2: )e difference between machine learning and deep Challenge. )e limited features used to model pedestrian learning [8]. behaviors and the need to apply metaheuristic algorithms to solve complex intelligent computing problems are among the major challenges in this research to model pedestrian (ii) Supervised or unsupervised learning: here, supervised behaviors. and unsupervised learning are linked to the class goal Kim et al. [10] examined pedestrian identification in label. Its availability means a supervised system, and smart buildings. Because the identification of pedestrians its absence indicates an unattended system. due to noise in images and some environmental factors and parameters faces challenges. )e researchers used the Deep 4. Using Deep Learning to Diagnose Pedestrians Convolution Neural Network (CNN) to create a vision- based model and the optimized version of the VGG-16, In today’s world, where the development of smart cities and called the OVGG-16, as the architectural core used to dis- smart transportation has received a lot of attention from tinguish pedestrians from the multitude of possible images. people, governments, and commercial and manufacturing To evaluate the proposed method, the researchers used the companies, one of the basic needs is to provide solutions to INRIA Dataset (http://pascal.inrialpes.fr/data/human/ identify objects around us by sensors and perform appro- (accessed on December 2020)), which contained 6817 im- priate operations according to movements performed by ages with 3239 pedestrian images, and the image quality in objects. Since in this research we have mainly focused on the this dataset was 227 × 227 pixels. )e results of the re- development of smart transportation in smart cities, so we searchers’ studies showed that the proposed method has a will focus only on identifying pedestrians who are influential high accuracy (approximately 98.8%) for the correct iden- in the development of smart cars and smart transportation, tification of pedestrians compared to other methods of and studies conducted by various researchers. In the field of machine learning. pedestrian identification, we have divided these studies into several groups, examining the studies related to each group Challenge. )e model created on noisy data has not been separately and pointing out the challenges in each. evaluated, and there is a question: if the set of images and input data has a lot of noise, how accurate will the pedestrian 4.1. Studies Conducted in the Field of Pedestrian Identification be identified by this proposed model? in Smart Cities. Belhadi et al. [9] studied the unusual be- Using deep learning, Tome` et al. [11] proposed a system for haviors of pedestrians in smart cities. For this purpose, pedestrian identification. )e researchers also proposed a new several algorithms were proposed, which are basically di- framework for identifying pedestrians. )e researchers also vided into two categories based on performance: proposed new solutions for different stages of pedestrian de- tection, which used deep learning to easily implement their (i) Algorithms that used different data mining and proposed algorithm on modern hardware. To implement and knowledge discovery techniques to discover the evaluate the proposed methods and solutions, they used the relationship between different behaviors of pedes- NVIDIA Jetson TK1, a GPU-based computing platform trians, and finally the knowledge generated to (https://developer.nvidia.com/embedded/jetson-tk1-developer- identify abnormal behaviors of pedestrians kit (accessed on December 2020)), and the Caltech Pedestrian (ii) Algorithms that have been developed based on the dataset (http://www.vision.caltech.edu/Image_Datasets/ history of pedestrian behaviors and based on dif- CaltechPedestrians/ (accessed on December 2020)). )is ferent characteristics of the user to detect abnormal dataset contains about 10 hours of video content related pedestrian behaviors to vehicles collected in different weather conditions. )is To implement these proposed algorithms, the re- dataset had 250 k frames per 137 minutes of video searchers used the HUMBI dataset (https://humbi-data.net/ content with 2300 different pedestrians. Half of the (accessed on December 2020)), which contains 164 attri- frames had no pedestrians, and 30% of the frames had 2 or 3 pedestrians. )e results of the implementation of butes (including gender, age, and physical condition) that include five basic body parts (including face, hands, body, these researchers showed that their proposed method has high efficiency and accuracy in identifying pedestrians in clothes, and eyes), which were designed using the data in this dataset in this study. real time. Journal of Advanced Transportation 7 )is program uses object classification and pedestrian Challenge. To implement these methods, we need large amounts of data, and the more data the dataset uses, the identification and location tracking. )e TensorFlow deep learning framework, Nvidia, cuDNN, and OpenCv accel- more efficient and accurate the proposed method will be. )e challenge arises when data collection for various reasons eration libraries, and the Caltech dataset were used to im- (including privacy) may not be possible in the metropolitan plement, learn, and test the proposed method. )is program areas of many countries in high-traffic urban areas. is installed for deployment in mobile phones or Embedded Systems connected to self-driving cars in order to develop driver assistance systems. 4.2. Studies Conducted in the Field of Pedestrian Identification for the Development of Intelligent Transportation Systems and Challenge. Real-time and accurate detection of objects (such Self-Driving Cars. Chen et al. [12] examined existing ar- as pedestrians) is one of the major challenges in the auto- chitectures for pedestrian detection when using the auto- motive industry to create and develop self-driving cars. With mated driving method. )ese researchers first explained the all the efforts that have been made, the percentage of pe- need to use methods to identify a pedestrian and determine destrian detection accuracy and the speed of detection of his or her route and then discussed the process of identifying existing methods are not enough, so these methods are not a pedestrian while driving a car. )ey, then, discussed how to very acceptable for applying real-time responses. use deep learning techniques (such as R-CNN, SVM) to Ahmed et al. [14] first compared the methods and discover two-step and one-step patterns and test the ef- techniques used to diagnose pedestrians and cyclists. )ey fectiveness of the patterns discovered to identify pedestrians. stated that because of, in the detection stage, the possibility Finally, the researchers examined and compared methods of detecting and locating objects (using deep learning proposed by other researchers to identify pedestrians. )ey techniques such as fast region-convolutional neural network also introduced several datasets (such as KTH, the UCF (R-CNN), faster R-CNN, and single shot detector (SSD)) in series, Hollywood2, and Google AVA) that are used to images and video frames, so the detection stage can be examine proposed methods for detecting pedestrian created as a vital part in creating and developing smart movement. applications in a self-driving vehicle. Finally, tracking results can be used to monitor and identify pedestrians or cyclists. Challenges. In this research, several important challenges in )e main purpose of this study was to investigate the existing identifying pedestrians are mentioned, which are as follows: methods for identifying cyclists. )e results of their studies (i) )e complexity of the environment around the showed that the use of appropriate techniques (e.g., sensor pedestrian can overshadow the operations and fusion and intent estimation) for identifying pedestrians and methods of recognizing the pedestrian and his cyclists can be an important step in maintaining road safety. In this research, first, the challenges in identifying and es- movement and, as a result, make it difficult to ac- curately identify the pedestrian. )erefore, creating timating the purpose and destination are presented, then a methods to identify different perspectives on pe- history of methods proposed by various researchers for destrian detection and operations performed by him pedestrian detection is presented, and the general steps is one of the challenges mentioned in this research. proposed for object detection are explained. Next, the re- search conducted by other researchers on the use of deep (ii) Pedestrian coverage can be extremely effective in the learning techniques and architectures to identify pedestrians process of identifying him/her. If the images are is reviewed, and then, the dataset used by various researchers taken from one perspective, this can affect the ac- to implement their proposed methods for pedestrian and curacy of pedestrian identification and reduce the cyclist detection is explained. accuracy of identification. )erefore, it is necessary for researchers to propose new methods for pre- Challenge. Most of the existing datasets for implementing paring multidimensional images and their simul- object detection techniques are focused on pedestrian de- taneous study and aggregation of the results for tection data, and there is no dedicated dataset to implement early identification of pedestrians, especially in self- the proposed techniques for identifying cyclists, so collecting driving cars. this type of dataset in different areas is currently a challenge. (iii) At present, there is no standard for determining the Zhu et al. [15] studied the challenges of pedestrian operations and actions in a vehicle against various detection using infrared and proposed to use deep movements performed by pedestrians. Better results learning methods for pedestrian detection to overcome can be obtained from the effects of identifying these challenges. By combining deep learning and pedestrians (such as monitoring the safety of pas- background subtraction methods, the researchers pro- sengers and the driver while traveling and managing posed a new method for pedestrian detection. )e pro- environmental pollution in cities) by creating a posed algorithm had two steps for pedestrian detection, classification and stating more details about driving which are as follows: practices in self-driving cars. Said and Barr [13] proposed a new program using deep Step 1: background subtraction methods are performed learning algorithm for fast and accurate pedestrian detection to provide information between frames for the machine to provide real-time responses in driver assistance systems. learning module 8 Journal of Advanced Transportation Challenge. )e dataset used in this study had a very small Step 2: refine Det equipment with a module of attention that is used to improve the accuracy of identifying number of records. )erefore, it seems that a larger dataset can be used to get better results. Also, this method should be pedestrians who are small in stature tested to identify other objects (such as cars and cyclists), In this study, a dataset consisting of infrared videos was and its accuracy should be checked to identify those objects. created that was used to identify pedestrians from a distance Dinakaran et al. [19] proposed generative adversarial and had good performance. networks (GANs) to create a new Cascaded Single Shot Detector (SSD) architecture for remote pedestrian detection. Challenge. )e proposed method in this research is based on In this architecture, DCGAN is used to improve the image a dataset created by the researcher, and its performance is quality for remote pedestrian detection. In this proposed well evaluated. )is is a set of video data stored by infrared. method, several criteria are used to identify the objects in the In order to prove the effectiveness of this method, it seems image. To implement the proposed method, the dataset of that it is necessary to use other datasets that have been the Canadian Institute for Advanced Research (CIFAR) is collected from different geographical locations with different used. )e results obtained from experiments have shown volumes of pedestrian traffic. that the proposed method has a high accuracy in identifying Bunel et al. [16] focused on remote pedestrian detection. vehicles and pedestrians from a distance. When the pedestrian is too far from the camera, the size of Challenge. Generative adversarial networks (GANs) can the pedestrian becomes very small, so it becomes very be used to remotely detect objects in smart cities. Since difficult to detect. )ese researchers suggested a neural security in communications created in IoT-connected net- network-based method and convolutional neural network- works in smart cities is very important and fundamental, so based method to learn the features with an end-to-end we conduct research on the use of GANs in improving approach to identify pedestrians who are too far away from security in smart cities to identify vehicles and pedestrians. the camera and too visible. Further, in this research, to Immediately, it seems very necessary. implement the proposed method, they used Caltech Pe- To overcome the problem of Occlusion handling, Tian destrian Datasets. )e results showed that the proposed et al. [20] proposed DeepParts, which consists of extensive method has a good performance in identifying pedestrians. trackers, instead of using deep learning techniques with an image detector. Some of the features of DeepParts are as Challenge. It seems that the quality of cameras and the follows: amount of pedestrian distance from the camera can be ef- fective in the performance of the proposed method and the (i) First, these DeepParts can be trained with poorly labeled data accuracy of method detection, which has not been con- sidered in this study. )erefore, it is recommended to ac- (ii) Second, DeepParts is able to handle low IoU positive curately define a standard for the best image quality and proposals that shift away from ground truth accuracy of diagnoses with different measurements and (iii) )ird, every part detector in DeepParts is a powerful different photographs or videos. tracker that can detect a pedestrian by observing Haghighat et al. [17] investigated the application of deep only a part of the body learning models in intelligent transportation systems. In the following, the advantages and disadvantages of embedded To implement the proposed method in this research, systems were discussed, and finally, the use of deep learning Caltech and KITTI datasets were used, and its performance techniques to predict the occurrence of traffic on different is compared to other detectors used for pedestrian detection. road routes was examined. Challenge. It is expected that, by using the combination of the results obtained from all the detectors used in the Challenge. All datasets used in research conducted by different parts, the accuracy of identifying objects, especially pedes- researchers have been collected using cameras installed in trians, will be increased. )e use of deep learning techniques different areas. It is expected that, with the advances made in and other techniques on data from detectors may improve the production of self-driving cars and sensors used on cars or pedestrian detection accuracy. on the street floor, the volume of data collected from them will In Navarro Lorente et al. [21], an automated sensor- be greatly increased, so we need new techniques in deep based system was used in applications on self-driving ve- learning to be able to analyze this data. hicles to identify pedestrians. Different types of sensors are Yu et al. [18] proposed a system for tracking and used in self-propelled vehicles, but in this study, researchers identifying pedestrians using deep neural networks, which focused on the Velodyne HDL-64E LIDAR sensor. )e data used a UAV and Kalman Filter forecasting method to track generated by this sensor was analyzed in three dimensions objects and pedestrians, and a dataset (YOLOv3) was used to using machine vision and machine learning algorithms implement the proposed method. To measure the efficiency (such as nearest neighbor algorithm, Bayesian classification of the proposed method for tracking and identifying pe- and support vector machine). A new framework called the destrians, accuracy and execution time and observing and Renault Twizy platform was proposed in this study to de- identifying objects were examined. )e results of experi- velop the ability of self-driving vehicles to identify pedes- mental experiments showed that the proposed method had trians. )e results of the implementation of their proposed fewer errors in identifying pedestrians. framework showed that their selected features along with the Journal of Advanced Transportation 9 cascade classification were used to achieve vehicle detection. algorithms used and the quality of the camera can be im- portant factors in better identifying pedestrians and )e performance evaluation results of the proposed method were about 90% for pedestrians and 88% for vehicle motorcyclists. Challenge. Implementing the algorithms used is time- detection. consuming, and it is necessary to propose methods for accurate and real-time identification of pedestrians and motorcyclists. 4.3. Proposed Methods for Pedestrian Detection Using Dif- Combs et al. [22] focused on using sensors installed on ferent Techniques. Cai et al. [25], to solve the problems self-driving cars to reduce the number of deaths due to caused by resizing objects at the accuracy and speed of object human-caused traffic accidents. )ey used the Fatality identification, provide a deep unified neural network, rep- Analysis Reporting System (FARS) to track the number of resenting the multicast CNN (MS-CNN), for the rapid human error deaths on US urban and suburban roads. )e detection of multifunctional objects. )ey gave MS-CNN researchers hypothesized that a car was traveling on a road including a proposed subnet and an identification subnet. and had all the necessary sensors to detect a pedestrian and )e proposed subnet has several output layers, in which fully effective software to detect and analyze the movements objects are detected at different scales. )e detection subnet and movements made by a pedestrian to identify that pe- uses tracking methods for multipurpose object monitoring. destrian. In addition, sensors mounted on the vehicle itself )e proposed method was implemented on the KITTI and are able to receive signals from the movement of pedestrians. Caltech datasets, and the results showed that the proposed As a result, a model can be developed to be able to easily method has a very good performance in detecting objects identify pedestrians and prevent accidents. )e proposed with a maximum of 15 frames per second. model used data from VLC cameras, radar-based detection systems, and light amplitude detection (LiDAR). )e results Challenge. In this research, the CNN feature approximation of their practical tests showed that, by using these facilities has been used as an alternative to input sampling. )e along with sensors installed on the car body, 90% of acci- challenge is whether other methods can be used to sample dents caused by human error can be prevented, while using the inputs that save more memory and time for calculations. only VLC can reduce the accident statistics by only 30%. Is it possible to increase the speed of moving and replacing Challenge. )ere is high cost of using sensors, VLC. frames (above 15 frames per second)? LiDAR and car-based radar detection systems prevent au- Fukui et al. [26] used complex neural network-based tomakers from using all of them to prevent rising car prices, (CNN) methods that are highly accurate in a variety of or from using low-quality cameras, which could be a reason contexts to identify pedestrians. )e researchers proposed a to reduce its quality and ultimately reduce the accuracy of new method proposed in this research based on CNN and identifying pedestrians or obstacles on the road. used Random Dropout and Ensemble Inference Network Song et al. [23] proposed an algorithm for detecting (EIN) for training and classification, respectively. Random pedestrians on the road. In this study, the pedestrian target Dropout selects units that have a high and variable flexibility area and the results of pedestrian detection on the road by rate for training, while, in a typical dropout, the flexibility combining the algorithm most similar to the neighbor and rate is fixed. EIN creates multiple networks with different the least energy algorithm were accurately divided. In this structures in well-connected layers. )e researchers used the study, all objects that are around a car and can generate Caltech and Daimler Mono pedestrian datasets to imple- traffic for pedestrian identification (such as cyclists, trees, ment their proposed method. other cars, and buildings around cars) are divided. And then, an algorithm was proposed for the environmental coverage Challenge. )e costs of real-time calculations to identify of the road. )e researchers used several experiments to pedestrians using this proposed method are relatively high, evaluate the performance of the traffic-generating object so it is necessary to adopt methods to reduce these costs. classification detection system proposed in this study. )ey To achieve better performance in applying deep learning selected and examined several sequences of images, in- theory to pedestrian detection, Cai et al. [27] improved the cluding different road scenes, different weather conditions, performance of a poorly supervised hierarchical deep and different city streets. learning algorithm with two-dimensional deep belief net- Challenge. )e time required to identify each of the works. In the proposed design of this research, the weak- obstacles (especially pedestrians) is variable, but in changing nesses of the structure and training methods used in the weather conditions and different road conditions, the time various algorithms of the existing classifications are iden- to identify obstacles and pedestrians can be increased or tified, and the following operations are performed to decreased. )erefore, we need to optimize the proposed eliminate these weaknesses: algorithm in this research to realize response time and (i) First, a network of deep one-dimensional beliefs identify pedestrians. expands to two-dimensional, allowing the image Hbaieb et al. [24] proposed a new method for detecting matrix to be loaded directly to preserve more in- the presence of pedestrians in the path of self-driving cars formation from the sample space. through an intervehicle communication system. In this method, descriptor (HOG), support vector machine clas- (ii) Second, a lightweight regulation term is added to sification (SVM), pedestrian tracker, and feature-based performance consistent with the training goal 10 Journal of Advanced Transportation able to identify pedestrians with high accuracy. )is rich without the use of traditional oversight. With this reform, the main training without supervision be- image database can be used in other detectors based on supervised learning architecture. comes weak training under supervision. (iii) )ird, the ability to distinguish between these Challenge. Creating datasets with a number of effective extracted features is created. features for pedestrian detection is one of the important (iv) In this research, the INRIA, Daimler, and CVC challenges that the more datasets we use have a variety of datasets of Spain have been used to implement and effective features for pedestrian detection and can more evaluate the accuracy of the proposed method. accurately identify pedestrians, using the method proposed in this study. Challenge. Working with unstructured data with existing Zeng et al. [33] first focused on in-depth collective public learning about each of the factors used to identify pedes- traditional methods faces several challenges (including challenges in the preprocessing, analysis, and grouping trians using advances in creating a new deep neural network architecture. )e proposed architecture in this research has stages). It is necessary to adopt methods or algorithms to optimize the performance of existing methods for analyzing the following parts: that data and identifying pedestrians through the results of (i) Filtered information maps are obtained from the those analyzes. It is also necessary to adopt strategies to first convolution layer. improve the performance of algorithms and methods used to (ii) From the second convolution layer, maps are ob- classify data and semantic information in occlusion tained to identify parts of the image. conditions. (iii) )e results obtained by identifying each part of the Saeidi and Ahmadi [28] first examined some of the pedestrian body are used to track maps and work DCNN-based learning methods and briefly explained the with information obtained from layers. Argument new algorithms proposed by various researchers for these about access to 20 feature parts or parts of the methods. Next, the researchers proposed a deep architec- pedestrian body is used to estimate the tag (for tural method and a new training method based on parallel example: does a particular window have a pedes- DCNNs for pedestrian detection. )e proposed method had trian or not?). two stages of training, which are as follows: (iv) )e windows are provided in dimensions (height 84 (i) Learning Candidate Pedestrian Extractor Network (CPEN) Candidate for pedestrian training and width 28) that the dimensions of the pedestrian can be identified by 60 by 20. (ii) Parallel training DCNNs (PDCNNs) to teach how to identify a candidate pedestrian by identifying the In other words, the proposed method in this research has body parts of that candidate pedestrian four parts for pedestrian detection, which are feature ex- traction, handling deformation, handling of occasions, and In this study, the Caltech-USA dataset was used to classification. implement the proposed method. )e results obtained from )e proposed method and architecture were imple- evaluating the accuracy of the proposed method in pedes- mented using Caltech and ETH datasets, and their efficiency trian detection and comparing it with other methods showed and accuracy in pedestrian identification were compared that this method has a higher accuracy compared to other with the accuracy of other deep learning methods. )e re- methods. sults show that the accuracy of the proposed method in this research is higher than that of other methods. Challenge. Selecting features for pedestrian detection, es- pecially in multidimensional data, is one of the most im- Challenge. To extract the effective features in high-precision portant challenges when using deep learning techniques. pedestrian detection, we need a large dataset with a large Deep learning techniques (such as SquaresChnFtrs, Infor- number of features that were not available in this study; so, medHaar, and Katamari) performed poorly in selecting to ensure the accuracy of the proposed method in this study, effective features for pedestrian detection, but deep learning we need to prepare a very large dataset with more features. techniques have recently been proposed by various re- Tarchoun et al. [34] proposed two methods for tracking searchers (e.g., CompAct-Deep [29], DeepParts [20], and pedestrians in images taken from moving vehicles: TA-CNN [30]). )ey performed much better in selecting suitable features for pedestrian detection. (i) In the first method, the block matching algorithm Vasconcelos et al. [31] proposed an automated method and block matching features are used to identify for optimizing the efficiency of the training suite by creating pedestrians deformation and creating a local perspective. Using this (ii) )e second method uses a faster R-CNN detector to method, human figures can be identified in the existing detect pedestrians training set by applying monitoring scenarios. Experimental )e proposed methods were implemented using the I2V- results of applying this method to datasets that included a variety of data and images (selection of 16 features from the MVPD database, and the results showed that the first imageNet dataset [32]) showed that if these data were en- method was able to detect pedestrians in images obtained tered as input to a convolutional neural network, it will be from moving vehicles in less time but had a higher false Journal of Advanced Transportation 11 KITTI dataset to implement the proposed method in this positive rate compared to the second method. )e second method had better accuracy and performance in pedestrian research. )e results of the evaluation of the proposed method showed that this method reduces the complexity detection. of the detectors and can be more efficient in accurately Challenge. Neither of these two methods can be used for real- identifying pedestrians. time pedestrian detection applications, so more research is needed to reduce costs and time on these two methods. Challenge. )e proposed method in terms of time required Lee et al. [35] proposed a deep fusion network-based to identify pedestrians may be associated with challenges; pedestrian detection method that used a single shot mul- i.e., in terms of time, more studies should be done on this tibox detector (DSSD) halfway through. )ey use correla- method, so that it can be used immediately to detect pe- tions between other feature maps to create new properties. destrians in cars used. Wagner et al. [38] explored the potential of deep learning In this study, deep fusion network was used to form issues related to the method of recognizing color images at night or techniques in pedestrian identification. )ey examined two deep fusion architectures and their performance on multi- pedestrian images in the dark. KAIST dataset was used to implement the proposed method. )e results obtained from spectral data. Finally, they used a new deep CNN-based the implementation and evaluation of the results showed method to detect pedestrians based on multispectral image that the proposed method, compared to other methods, had data to analyze the proposed method. )ey introduced the at least 4.28% lower error rate in identifying pedestrians in first deep CNN application for pedestrian detection based on the dark environments. multispectral image data, and they used three datasets (including ImageNet [32], CALTECH benchmark [2], and Challenge. Correctly identifying and exacting location of KAIST) to implement and evaluate the proposed method. pedestrians in the dark using existing methods is still a )e evaluation results showed that the proposed method had a higher accuracy in pedestrian detection compared to other challenge. Creating ways to connect different features and deep learning techniques can go a long way in increasing the methods. accuracy of identifying pedestrians in the dark. Ribeiro et al. [36] proposed a deep learning method for Challenge. )e most important challenge in the proposed pedestrian detection (PD) detection in real time to solve method is that, most of the time, early-fusion architecture is problems related to the human-aware robot navigation not able to achieve our expected performance. )e reason for problem. To achieve fast and accurate pedestrian detection this may be due to the inability of the early-fusion network to efficiency, this study developed a combination of Aggregate learn the meaningful multistate abstract properties in a given Channel Features (ACF) detector with a deep convolutional environment. Kim et al. [39] proposed a system with limited re- neural network (CNN). In this method, we have tried to use CNN to increase the accuracy of pedestrian detection by sources for real-world monitoring and identification of moving persons. For this purpose of combination back- trackers. To implement the proposed method and evaluate its accuracy, two sets (called corridor and Mbot) were used, ground subtraction and convolutional neural networks (CNNs), they used it to identify and detect moving objects which have real photos taken by the cameras (photos col- lected from the cameras in the internal and external sensors using outdoor CCTV videos. )e background subtraction of the robot), and a typical robot navigation environment algorithm used to find the desired areas in the video frame was used to evaluate the accuracy of the method in iden- and the CNN classifier was used to classify the ROIs tifying pedestrians, and the results showed that it has suf- obtained in one of the predefined classes. To implement ficient speed and accuracy to be used in these environments the proposed method in practice, various datasets collected and robot navigation applications to identify pedestrians. by several real-world CCTV cameras were used. )e re- sults showed that the proposed system had a high accuracy Challenge. )e performance of the proposed method should in identifying pedestrians and was also less complex than other methods. be evaluated on datasets collected from cameras located in different places with different light intensities and distances, Challenge. Occurrence of some problems in the collected different types of sensors installed in the environment such as laser sensors. data can reduce the performance or accuracy of pedestrian Hu et al. [37] worked to create a powerful pedestrian detection. For example, lack of training data may disrupt the detector. For this purpose, the researchers designed the training process. On the other hand, using the same images deep convolutional neural network (DCNN) as an image will cause a pedestrian to be repeatedly identified several feature to teach a set of enhanced decision models, using times, and this will reduce the performance of the proposed redesigned learning algorithms (CFMs) without the use of system for pedestrian detection. Lin et al. [40] proposed a framework for pedestrian learning algorithms. To increase the efficiency and accu- racy of DCNN-based detectors for image detection of detection that is based on incorporating pixel-wise infor- mation into deep convolutional feature maps. In this con- pedestrians, hand-crafted features such as optical flow are used. In this study, they reviewed various datasets that text, they used the zooming properties to improve image have been used by other researchers to implement their quality to help easily and accurately identify pedestrians. proposed methods for pedestrian detection. )ey used the )erefore, the proposed method in this research helps 12 Journal of Advanced Transportation Table 1: Challenges and proposed solutions. Challenges Proposed solutions Installation of devices in the desired routes to monitor the passage of Lack of structured data in all studied routes to identify pedestrians pedestrians and bicycles around the clock and the use of various data at different times of the day with different light intensities during mining techniques and deep learning to create patterns to identify the day and different weather conditions cyclists and pedestrians in different weather conditions and different light intensities Absence of any known standard for selecting appropriate and essential features from the features collected for pedestrian Using more advanced sensors in different directions or applying identification or limited features used to model pedestrian sensors in the car body, creating a centralized database to consolidate behaviors and the need to apply metaheuristic algorithms to solve data collected from sources and sensors used in different places complex intelligent computing problems Creating high-precision and high-quality sensors and cameras, Existence of noise in various images and data collected, poor quality installing high-quality cameras on sensitive routes, proposing new or very low quality of some images collected by cameras installed on techniques for preprocessing data and images, and accurately the road, especially in cloudy, rainy and icy weather or dark at night detecting noise images Establish protocols and standards for collecting data on public places, Lack of access to clear and uninterrupted data and images for establish agreements and laws to protect the privacy of the public research due to privacy while collecting 24-hour data from all road routes )e complexity of the environment around the pedestrian can overshadow the operations and methods of identifying the Develop methods to identify different perspectives on pedestrian pedestrian and her movement, and as a result, make it difficult to detection and operations performed by his/her accurately identify the pedestrian Pedestrian coverage can be very effective in the process of Proposing new methods by researchers to prepare multidimensional identifying him. If the images are taken from one perspective, it can images and their simultaneous study and aggregation of the results affect the accuracy of pedestrian identification and reduce the for early identification of pedestrians, especially in cars accuracy of identification Real-time and accurate detection of objects (such as pedestrians) is Propose methods to increase the accuracy of pedestrian detection and an important challenge for car companies to create and develop speed up the process of immediate response to avoid accidents when self-driving cars detecting obstacles and pedestrians Collect data on cyclists on different routes with different light Lack of specific data to diagnose cyclists intensities and create patterns to identify cyclists )e quality of the cameras, the amount of pedestrian distance from the camera can affect the performance of the proposed method and Define a standard for the best image quality and detection accuracy the accuracy of the method detection, which has not been with different measurements and different shots or videos considered in many studies )e cost of performing real-time calculations to identify It is necessary to adopt methods to reduce these costs pedestrians using this proposed method is relatively high It is necessary to adopt methods or algorithms to optimize the performance of existing methods for analyzing that data and Working with unstructured data with existing traditional methods identifying pedestrians through the results of those analyzes. It is also faces several challenges (including the challenges of preprocessing, necessary to adopt strategies to improve the performance of analysis, and grouping) algorithms and methods used to classify data and semantic information in occlusion conditions identify pedestrians who are seen in a very small image by the performance of these detectors using various datasets inserting geographical location specifications and pedestrian including Caltech. In this study, the most widely used features. )e proposed method uses three datasets: Caltech datasets were briefly described, and the strengths and [41], INRIA [42], and KITTI [43]. )e implementation weaknesses of each were expressed. )ree features (in- results obtained from the evaluation and comparison with cluding best features, additional data, and background/ other methods showed that this method is more efficient in conceptual information) were used to conduct practical terms of reducing the time of pedestrian identification and experiments in this study, which can affect the efficiency of the proposed method for pedestrian detection. )ree im- the number of unidentified cases. portant and famous trackers (including deformable part Challenge. Due to the small size of the pedestrian image, it models, decision forests, and deep networks) are based on seems that there are complications in recognizing of that the different learning techniques used. pedestrian in an image taken from a low light environment, especially at night, using the method proposed in this research. Challenge. It seems that the most important challenge in the Dollar ´ et al. [41] reviewed advances over the past decade field of pedestrian detection is to develop a deeper under- in developing methods for pedestrian detection and pro- standing in selecting the best features to achieve the highest posing 40 trackers for pedestrian detection. )ey analyzed accuracy and performance in real-time pedestrian detection. Journal of Advanced Transportation 13 accessible surface area of proteins by iterative deep learning,” )e main challenge ahead seems to develop a deeper un- Scientific Reports, vol. 5, no. 1, p. 11476, 2015. derstanding of what makes good features, so as to enable the [8] S. Dargan and M. Kumar, “A survey of deep learning and its design of even better ones. applications: a new paradigm to machine learning,” Archives of Computational Methods in Engineering, vol. 27, pp. 1071– 5. Conclusion 1092, 2019. [9] A. Belhadi, Y. Djenouri, G. Srivastava, D. Djenouri, Nowadays, the amount of data generated during a day from J. C.-W. Lin, and G. Fortino, “Deep learning for pedestrian various sensors and other devices is enormous, and collective behavior analysis in smart cities: a model of group technologies such as cloud computing are being used to trajectory outlier detection,” Information Fusion, vol. 65, help store large volumes of collected data. One of the pp. 13–20, 2021. benefits of analyzing this data is the discovery of knowledge [10] B. Kim, N. Yuvaraj, K. R. Sri Preethaa, R. Santhosh, and and a pattern for use in similar situations by training A. Sabari, “Enhanced pedestrian detection using optimized deep convolution neural network for smart building sur- machines. In this study, we explained deep learning and its veillance,” Soft Computing, vol. 24, no. 22, pp. 17081–17092, difference with machine learning, and then some research is done by various researchers around the use of deep [11] D. Tome, ` F. Monti, L. Baroffio, L. Bondi, M. Tagliasacchi, and learning techniques in the creation and development of S. Tubaro, “Deep convolutional neural networks for pedes- smart cities and pedestrian identification in smart cities trian detection,” Signal Processing: Image Communication, and Intelligent Transportation Systems (ITS). Finally, we vol. 47, pp. 482–489, 2016. examined smart transportation and listed the challenges in [12] L. Chen, N. Ma, P. Wang et al., “Survey of pedestrian action each of them. recognition techniques for autonomous driving,” Tsinghua In general, according to the studies conducted in this Science and Technology, vol. 25, no. 4, pp. 458–470, 2020. paper, the most important challenges in identifying pe- [13] Y. F. Said and M. Barr, “Pedestrian detection for advanced destrians on the street using the proposed technologies and driver assistance systems using deep learning algorithms,” methods, especially deep learning techniques, can be IJCSNS (International Journal of Computer Science and Network Security), vol. 19, no. 10, 2019. expressed in Table 1 and some of the solutions that seem [14] S. Ahmed, M. N. Huda, S. Rajbhandari, C. Saha, M. Elshaw, useful in solving these challenges are suggested in this table. and S. Kanarachos, “Pedestrian and cyclist detection and intent estimation for autonomous vehicles: a survey,” Applied Data Availability Sciences, vol. 9, no. 11, p. 2335, 2019. [15] Y. Zhu, J. Yang, X. Xieg, Z. Wang, and X. Deng, “Long- )is is a survey, and we introduced already all databases in distanceinfrared video pedestrian detection using deep this study. learning and backgroundsubtraction,” Journal of Physics: Conference Series, vol. 1682, 2020. [16] R. Bunel, F. Davoine, and P. Xu, “Detection of pedestrians at Conflicts of Interest far distance,” in Proceedings of the 2016 IEEE International )e authors declare that they have no conflicts of interest. Conference on Robotics and Automation (ICRA), IEEE, Stockholm, Sweden, May 2016. [17] A. K. Haghighat, V. Ravichandra-Mouli, P. Chakraborty, References Y. Esfandiari, S. Arabi, and A. Sharma, “Applications of deep learning in intelligent transportation systems,” Journal of Big [1] R. E. Schapire and Y. Freund, Foundations of Machine Data Analytics in Transportation, vol. 2, no. 2, pp. 115–145, Learning, MIT Press, Cambridge, MA, USA, 2012. [2] ETSI, “Intelligent transport systems (ITS); security; threat, [18] H. Yu, G. Li, W. Zhang et al., “)e unmanned aerial vehicle vulnerability and risk analysis (TVRA),” European Tele- benchmark: object detection, tracking and baseline,” Inter- communications Standards, Sophia Antipolis, France, ETSI national Journal of Computer Vision, vol. 128, no. 5, TR 102 893, 2010. pp. 1141–1159, 2020. [3] T. Rawal and V. Devadas, “Intelligent transportation system [19] R. K. Dinakaran, P. Easom-McCaldin, A. Bouridane et al., in India—a review,” Journal of Management Development, “Deep learning based pedestrian detection at distance in smart vol. 2, p. 299, 2015. cities,” in Intelligent Systems and Applications, Springer, [4] P. Fernandes and U. Nunes, “Vehicle communications: a Cham, Germany, 2019. short survey,” in Proceedings of the IADIS Telecommunica- [20] Y. Tian, P. Luo, X. Wang, and X. Tang, “Deep learning strong tions, Networks and Systems, p. 1556, Lisboa, Portugal, July parts for pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, [5] K. B. Smith, “Typologies, taxonomies, and the benefits of policy classification,” Policy Studies Journal, vol. 30, no. 3, December 2015. [21] P. J. Navarro Lorente, C. Fernandez, R. Borraz, and D. Alonso, pp. 379–395, 2002. [6] T. Brosch, R. Tam, and Initiative for the Alzheimers Disease “A machine learning approach to pedestrian detection for autonomous vehicles using high-definition 3D range data,” Neuroimaging, “Manifold learning of brain MRIs by deep learning,” in Proceedings of the International Conference on Sensors, vol. 17, no. 1, p. 18, 2017. [22] T. S. Combs, L. S. Sandt, M. P. Clamann, and N. C. McDonald, Medical Image Computing and Computer-Assisted Intervention, Nagoya, Japan, September 2013. “Automated vehicles and pedestrian safety: exploring the promise and limits of pedestrian detection,” American Journal [7] R. Heffernan, K. Paliwal, J. Lyons et al., “Improving prediction of secondary structure, local backbone angles, and solvent of Preventive Medicine, vol. 56, no. 1, pp. 1–7, 2019. 14 Journal of Advanced Transportation [23] Y. Song, J. Yao, Y. Ju, Y. Jiang, and K. Du, “Automatic de- [37] Q. Hu, P. Wang, C. Shen, A. v. d. Hengel, and F. Porikli, tection and classification of road, car, and pedestrian using “Pushing the limits of deep CNNs for pedestrian detection,” IEEE Transactions on Circuits and Systems for Video Tech- binocular cameras in traffic scenes with a common frame- nology, vol. 28, no. 6, pp. 1358–1368, 2017. work,” Complexity, vol. 2020, Article ID 2435793, 17 pages, [38] J. Wagner, V. Fischer, M. Herman, and S. Behnke, “Multi- spectral pedestrian detection using deep fusion convolutional [24] A. Hbaieb, J. Rezgui, and L. Chaari, “Pedestrian detection neural networks,” in Proceedings of the 24th European Sym- for autonomous driving within cooperative communica- posium on Artificial Neural Networks, Computational Intel- tion system,” in Proceedings of the 2019 IEEE Wireless ligence and Machine Learning (ESANN), Bruges, Belgium, Communications and Networking Conference (WCNC), April 2016. IEEE, Marrakesh, Morocco, April 2019. [39] C. Kim, J. Lee, T. Kan, and Y.-M. Kim, “A hybrid framework [25] Z. Cai, Q. Fan, R. S. Feris, and N. Vasconcelos, “A unified multi- combining background subtraction and deep neural networks scale deep convolutional neural network for fast object detec- for rapid person detection,” Journal of Big Data, vol. 5, no. 1, tion,” in European Conference on Computer Vision, Springer, pp. 1–24, 2018. Cham, Germany, 2016. [40] C. Lin, J. Lu, G. Wang, and J. Zhou, “Graininess-aware deep [26] H. Fukui, T. Yamashita, Y. Yamauchi, H. Fujiyoshi, and feature learning for pedestrian detection,” in European H. Murase, “Pedestrian detection based on deep convolu- Conference on Computer Vision (ECCV), Springer, Cham, tional neural network with ensemble inference network,” in Germany, 2018. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium [41] P. Dollar, ´ C. Wojek, B. Schiele, and P. Perona, “Pedestrian (IV), IEEE, Seoul, South Korea, July 2015. detection: a benchmark,” in Proceedings of the 2009 IEEE [27] Y. Cai, Y. He, H. Wang, X. Sun, L. Chen, and H. Jiang, Conference on Computer Vision and Pattern Recognition, June “Pedestrian detection algorithm in traffic scene based on weakly supervised hierarchical deep model,” International [42] N. Dalal and B. Triggs, “Histograms of oriented gradients for Journal of Advanced Robotic Systems, vol. 14, no. 1, Article ID human detection,” in Proceedings of the 2005 IEEE Computer 1729881417692311, 2016. Society Conference on Computer Vision and Pattern Recog- [28] M. Saeidi and A. Ahmadi, “Deep learning based on parallel nition (CVPR’05), IEEE, San Diego, CA, USA, June 2005. CNNs for pedestrian detection,” International Journal of [43] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for au- Information and Communication Technology Research, vol. 10, tonomous driving? the kitti vision benchmark suite,” in no. 4, pp. 42–52, 2018. Proceedings of the 2012 IEEE Conference on Computer Vision [29] Z. Cai, M. Saberian, and N. Vasconcelos, “Learning com- and Pattern Recognition, IEEE, Providence, RI, USA, June plexity-aware cascades for deep pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, December 2015. [30] Y. Tian, P. Luo, X. Wang, and X. Tang, “Pedestrian detection aided by deep learning semantic tasks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015. [31] C. N. Vasconcelos, A. Paes, and A. Montenegro, “Towards deep learning invariant pedestrian detection by data en- richment,” in Proceedings of the 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), IEEE, Anaheim, CA, USA, December 2016. [32] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and F.-F. Li, “Imagenet: a large-scale hierarchical image database,” in Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, June 2009. [33] X. Zeng, W. Ouyang, and X. Wang, “Multi-stage contextual deep learning for pedestrian detection,” in Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, December 2013. [34] B. Tarchoun, A. Khalifa, S. Dhifallah, I. Jegham, and M. Mahjou, “Hand-crafted features vs deep learning for pedestrian detection in moving camera,” Traitement du Sig- nal, vol. 37, no. 2, pp. 209–216, 2020. [35] Y. Lee, T. D. Bui, and J. Shin, “Pedestrian detection based on deep fusion network using feature correlation,” in Proceedings of the 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, Honolulu, HI, USA, November 2018. [36] D. Ribeiro, R. Mateus, P. Miraldo, and J. C. Nascimento, “A real-time deep learning pedestrian detector for robot navi- gation,” in Proceedings of the 2017 IEEE International Con- ference on Autonomous Robot Systems and Competitions (ICARSC), IEEE, Coimbra, Portugal, April 2017.

Journal

Journal of Advanced TransportationHindawi Publishing Corporation

Published: Oct 4, 2021

There are no references for this article.