Perceive, Attend, and Drive:
Learning Spatial Attention for Safe Self-Driving
Abstract
In this paper, we propose an end-to-end self-driving network featuring a sparse attention module that learns to automatically attend to important regions of the input. The attention module specifically targets motion planning, whereas prior literature only applied attention in perception tasks. Learning an attention mask directly targeted for motion planning significantly improves the planner safety by performing more focused computation. Furthermore, visualizing the attention improves interpretability of end-to-end self-driving.
1 Introduction
Self-driving is one of today’s most impactful technological challenges, one that promises to bring safe and affordable transportation everywhere. Tremendous improvements have been made in self-driving perception systems, thanks to the success of deep learning. This has enabled accurate detection and localization of obstacles, providing a holistic understanding of the surrounding world, which is then sent to the motion planner to decide subsequent driving actions.
Despite the eminent success of these perception systems, their detection objective is mis-aligned with the self-driving vehicle’s overall goal—to drive safely to the destination. We typically train the perception systems to detect all objects in the sensor range, assigning each object an equal weight even if some objects are not important as they will never interact with the self-driving vehicle. For example, they could be far away or parked on the other side of the road as in Figure 1. As a result, a vast amount of computation and model capacity is wasted in recognizing very difficult instances that matter only for common metrics such as average precision (AP), but not so much for driving. This is in striking contrast with how humans drive: we focus our visual attention in areas that directly impact safe planning. Inspired by the use of visual attention in our brain, we aim to introduce attention to self-driving systems to efficiently and selectively process complex scenes.
Numerous studies in the past have explored adding sparse attention in deep neural networks to improve computation efficiency in classification [47, 8] and object detection [35, 11, 36]. In order to perform well on the metrics employed in common benchmarks, the attention mask in [35] still needs to cover all actors in the scene, slowing the network when the scene has many vehicles.
In this paper, our aim is to address these inconsistencies such that the amount of computation is optimized towards the end goal of motion planning. Specifically our contributions are as follows:
-
We learn an attention mask directly towards the motion planning objective for safe self-driving.
-
We use the attention mask to reweight object detection and motion forecasting losses in our joint end-to-end training, focusing the model capacity on objects that matter most. Different from manually prioritizing instances [34], here the weighting is entirely data-driven.
-
Our attention-based model achieves significantly reduced collision rate and improved planning performance, while at a much lower computation cost.
-
Attention mask visualization improves interpretability of end-to-end deep learning models in self-driving.
![](figures/intro_vis.png)
2 Related Work
Attention mechanism in deep learning: Human and other primate visual perception systems feature visual attention to reduce the complexity of the scene and speed-up inference [14, 13]. Earlier studies in visual saliency aimed to predict human gaze with no particular task in mind [17]. Attention mechanisms nowadays are built in as part of end-to-end models to optimize towards specific tasks. The attention modules are typically implemented as multiplicative gates to select features. This schema has shown to improve performance and interpretability on downstream tasks such as object recognition [1, 22, 47], instance segmentation [36], image captioning [49], question answering [29, 50], as well as other natural language processing applications [2, 46, 6]. The visualization of the end-to-end learned attention suggests that deep attention-based models have an intelligent understanding of the inputs by focusing on the most informative parts of the input.
Sparse activation in neural networks: Sparse coding models [31] use an overcomplete dictionary to achieve sparse activation in the feature space. In modern convolutional neural networks (CNNs), sparsity is typically brought by the widespread use of ReLU activation functions, but these are rather unstructured, and speed-up has only been shown on specially designed hardware [16, 42]. Structured spatial sparsity, on the other hand, can be made efficient by using a sparse convolution operator [9, 35, 10], which in turn allows the network to shift its focus on more difficult parts of the inputs [8, 23, 35, 20]. In self-driving, [34] proposed a ranking function to prioritize computations that would have the most impact on motion planning. Weight pruning [26, 28] is another popular way to achieve sparsity in the parameter space, which is an orthogonal direction to our method.
Attention and loss weighting in multi-task learning: Our end-to-end self-driving network is an instance of multi-task learning as all three tasks—perception, prediction and motion planning—are simultaneously solved by individual output branches with shared features. It is common to use a summation of all the loss functions, but sometimes there are conflicting objectives among the tasks. Prior literature in multi-task learning has studied dynamic weighting towards different loss components, by using training signals such as uncertainty [18], gradient norm [5], difficulty level [12], or entirely data-driven objectives [25, 37]. In [25, 37], task and example weights are learned by optimizing the performance of the main task. The attention mechanism has also been used in multi-task learning: in [27], a network applies task-specific attention masks on shared features to encourage the outputs to be more selective. Similar to dynamic loss weighting models [25], we exploit the learned attention towards weighting instance detection losses. Instead of using multiple attentions, as was done in [27], we use one single attention mask to optimize our main task: driving.
Safety-driven learnable motion planning: One of the primary motivations of introducing attention into an end-to-end motion planning network is to improve safety. Traditionally, safety for self-driving models was done in terms of formal model checking and validation [41, 45, 32, 33, 48]. More recently, with the widely available driving data, imitation learning has been introduced in self-driving to learn from cautious human driving [51, 7, 40, 39, 52]. Safety has also been considered in terms of explicitly learning a risk sensitive measure from human demonstration [44, 21]. In our work, although safety is not explicitly encoded in our loss function, we have experimentally verified that the sparse attention models are significantly better at avoiding collisions.
![](figures/nmp2.png)
3 Perceive, Attend and Drive
In this section, we present our framework for using learned, motion-planning aware attention. We first describe the end-to-end neural motion planner that serves as the starting point of our work, and then introduce our proposed attention module and attention-driven loss function, which enable us to focus the computation in areas that matter for the end task of driving.
![](figures/sa-nmp2.png)
3.1 A review on Neural Motion Planner (NMP)
Our proposed model extends upon the neural motion planner (NMP), which jointly solves the perception, prediction and planning problems for self-driving. In this section we briefly review NMP depicted in Figure 2, and refer the reader to [51] for more details.
Input and backbone: NMP voxelizes LiDAR point clouds to a birds-eye-view (BEV) feature map and fuses them with channels of rasterized HDMap features to produce an input representation of size , where are the height and spatial dimensions and is the number of input LiDAR sweeps. The backbone consists of 5 blocks, with the first 4 producing multi-scale features that are concatenated and fed to the final block. Overall the backbone downsamples the spatial dimension by 4.
Multi-task headers: Given the features computed by the backbone, , NMP uses two separate headers for perception & prediction, and motion planning. The perception & prediction header consists of separate branches for classification and regression. The classification branch outputs a score for each anchor box at each spatial location over the feature map , while the regression branch outputs regression targets for each anchor box, including targets for localization offset, size, and heading angle. The planning header consists of convolution and deconvolution layers to produce a cost volume representing the cost for the self-driving vehicle to be at each location and time, with the fixed future planning horizon.
Planning inference: At inference time, NMP samples trajectories that are physically realizable, and chooses the lowest cost trajectory for the ego-car:
(1) |
where the cost of a trajectory is the sum of all its waypoints in the cost volume:
(2) |
3.2 Sparse Attention Neural Motion Planner (SA-NMP)
In this section, we propose our sparse spatial attention module for self-driving, shown in Figure 3, which learns to save computation while performing well on the end task of driving safely to the goal.
Input and backbone: We exploit the same input representation as NMP and use the same perception, prediction, and planning headers. The NMP backbone is replaced with the state-of-the-art backbone network of PnPNet [24], which uses cross-scale blocks throughout to fuse BEV sensory input. Each cross-scale block consists of 3 parallel branches at different resolutions that downsample the feature map, perform bulk computations, and then upsample back to the backbone resolution, before finally fusing cross-scale features across all branches. There is an additional residual connection across each cross-scale block. The final output feature from the backbone consists of 128 channels at 4x downsampled resolution, which is forwarded to the planning and detection headers. In addition to the improved performance, PnPNet can be easily scaled for different computational budgets by varying the depth and width of the cross-scale blocks.
Existing attention-driven approaches [35] tackle only the perception task and use either a road mask obtained from map information or a vehicle mask produced by a different perception module. As a consequence, they waste computation on areas that will not affect the self-driving car. We instead propose a novel approach that is end-to-end trainable and performs computation selectively for planning a safe maneuver. As shown in Fig. 3, the learned attention mask then gates the backbone network, limiting computation to areas where attention is active. By using binary attention, we can leverage sparse convolution to improve the computational efficiency.
Generating binary attention: Computational efficiency has been shown to be one of the most prominent advantages of using the attention mechanism. For soft attention masks the computation is still dense across the entire activation map, and therefore no computation savings can be achieved. SBNet [35] showed that a sparse convolution operator can achieve significant speed-ups with a given discrete binary attention mask. Here, we would also like to exploit the computational benefit of sparse convolution by using discrete attention outputs.
We utilize a network to predict a scalar score for each spatial location, and binarize the score to represent our sparse attention map. For efficiency and simplicity, we use a small U-Net [38] with skip connections and two downsample/upsample stages. We would like to apply the generated attention back to the BEV features in the model backbone so as to sparsify the spatial information, allowing computation to be focused on the important regions only. We choose to do so in a residual manner [47, 35] to avoid deteriorating the features throughout the backbone. Let denote the normal residual block. Our attention mask is multiplied with the input to the residual block as follows:
(3) |
where denotes elementwise multiplication. See Fig. 3 for an illustration of our architecture.
Learning binary attention with Gumbel softmax: In order to learn the attention generator and backpropagate through the binary attention map, we make use of the Gumbel softmax technique [15, 30] since the step function is not differentiable, and using the standard sigmoid function suffers from a more severe bias-variance trade-off [15]. Let denote spatial coordinates, and the scalar output from the attention U-Net. We first add the Gumbel noise on the logits as follows:
(4) | ||||
(5) | ||||
(6) |
where , and is sampled from . At inference time, hard attention can be obtained by comparing the logits,
(7) |
During training, however, we would like to approximate the gradient by using the straight-through estimator [15, 3]. Hence, in the backward pass, the step function is replaced with a softmax function with a temperature constant (where is the underlying soft attention):
(8) |
3.3 Multi-Task Learning
We train our sparse neural motion planner (including the attention) end-to-end with a multi-task learning objective that combines planning () with perception & motion forecasting (, ):
(9) |
where is an loss, defined in Equation 17, that controls the sparsity of the attention mask, and is the standard weight decay term. Following [51, 24], we fix .
Motion planning loss: The motion planning loss utilitizes the max-margin objective, where the ground-truth driving trajectory (performed by a human) should be of lower cost than other trajectories sampled by the model. Let be the groundtruth trajectory and let be the cost volume output by the model. We sample lowest cost trajectories based on the cost volume: , and penalize the maximum margin violation between the groundtruth and model samples:
(10) |
where is the task loss capturing spatial differences, and traffic violations encoded in binary:
(11) |
Perception & prediction (PnP) loss: This loss follows the standard classification and regression objectives for object detection. The classification part uses binary cross-entropy:
(12) |
where is the predicted classification score between 0 and 1, and is the binary groundtruth. For each detected instance, the model outputs a bounding box, and a pair of coordinates and angles for each future step. We reparameterize the shift of a bounding box from an anchor bounding box in a 6-dimensional vector :
(13) |
A regression loss is then applied for the trajectory of the instance up to time . For each spatial coordinate , we sum up the losses of all bounding boxes where the ground truth belongs to this location:
(14) |
with the model predicted shifts and the ground truth shifts.
Auxiliary loss masking: Our overall objective is to achieve good performance in motion planning, so PnP is an auxiliary task. Since the majority of our computation happens within the attended area, intuitively the model should not be penalized as severely for mis-detecting objects not in the attended area. We therefore propose to use our spatial attention mask to re-weight the PnP losses as follows:
(15) | ||||
(16) |
where weights attended instances, and weights all instances. We fix and .
Attention sparsity loss: To encourage focused attention and high sparsity, we use an regularizer on the attention mask as follows. We control sparsity with in Equation 9.
(17) |
![](figures/mainresult.png)
4 Experimental Evaluation
We evaluated on a real-world driving dataset (Drive4D), training on over 1 million frames from 5,000 scenarios and validating on 5,000 frames from 500 scenarios, using both LiDAR and HD-maps. We also evaluated on nuScenes v1.0 [4], a large-scale public dataset, with a training set of over 200,000 frames and a test set of 5,000 frames. Due to the inaccurate localization they provide, we omitted HDMaps and only used LiDAR [24].
4.1 Implementation Details and Metrics
Training: To jointly train SA-NMP with attention, we use pretrained weights for the backbone and headers from training a SA-NMP without attention (dense) for two epochs. We train all our models with batch size 5 across 16 GPUs in parallel using Adam [19] optimizer. We use an initial learning rate of 0.0001 and a decay of 0.1 at 1.0 and 1.6 epoch(s), for a total of another 2.0 epochs.
Evaluation: To evaluate driving and safety performance, we focus on the following planning metrics which are accumulated over all 6 future timesteps (3s): Planning L2 is the L2 distance between waypoints of the predicted future ego trajectory and those of the ground-truth trajectory (characterized by human driving). Collision rate is the frequency of collisions between the planned ego trajectory and the ground truth trajectories of other actors in the scene. Lane violation rate measures the number of lane boundary violations by the planned ego trajectory. we do not evaluate this on nuScenes due to the inaccurate localization they provide,
Baselines: We compare our learned attention to baselines that are end-to-end trained using static attention masks obtained from priors. Road Mask covers the entire road as provided from the map data. Vehicle Mask strictly covers all detections in the input space, obtained from a PSPNet [53] trained for segmentation. Proximity Mask is a circular radius around the ego vehicle. Dense is not using sparse attention.
Drive4D | Backbone | Sparsity | Planning L2 | Collision Rate | Lane Violation |
---|---|---|---|---|---|
Dataset | FLOPS | at 3s (m) | over 3s (%) | over 3s (%) | |
NMP [51] | 39.18B | 0.0% | 2.279 | 0.657 | 2.780 |
Dense SA-NMP | 22.73B | 0.0% | 2.207 | 0.639 | 1.350 |
SA-NMP+Vehicle Mask | 5.43B | 93.6% | 2.297 | 0.639 | 1.387 |
SA-NMP+Proximity Mask | 5.31B | 94.0% | 2.276 | 0.584 | 1.387 |
SA-NMP+Road Mask | 12.85B | 68.9% | 2.194 | 0.548 | 1.296 |
SA-NMP+Learned Attn (Ours) | 5.22B | 95.0% | 2.102 | 0.511 | 1.338 |
nuScenes | Backbone | Sparsity | Planning L2 | Collision Rate |
---|---|---|---|---|
Dataset | FLOPS | at 3s (m) | over 3s (%) | |
NMP [51] | 39.18B | 0.0% | 2.310 | 1.918 |
Dense SA-NMP | 22.73B | 0.0% | 2.271 | 2.198 |
SA-NMP+Vehicle Mask | 5.44B | 94.0% | 2.263 | 2.234 |
SA-NMP+Proximity Mask | 5.31B | 94.0% | 2.103 | 2.073 |
SA-NMP+Learned Attn (Ours) | 5.34B | 94.0% | 2.052 | 1.588 |
4.2 Results
Quantitative results: With a sparse attention mask learned towards motion planning, we can leverage the sparsity in the network backbone to greatly reduce computational costs, while not only maintaining but improving model performance. In our experimental results, we use theoretical FLOPs to show the efficiency of our network, but this also translates to real time gains as SBNet [35] has been shown to leverage sparsity to achieve real speed-ups. The increase in efficiency from leveraging sparsity is shown in Table 1, where our learned attention model uses fewer FLOPs than Dense SA-NMP thanks to its 95% sparse attention mask. Also, even with an identical SA-NMP backbone as the baselines (except NMP), our model with learned attention performs better in all motion planning metrics, which indicates that focused backbone computation is greatly advantageous to the overall goal of safe planning. NMP+Road performs slightly better in Lane Violation due to the road mask attention focusing on all road/lane markings. However, this baseline uses more than double the FLOPs since its attention mask looks at the entire road surface at only 68.9% sparsity. From Fig. 4, our learned attention model clearly outperforms other baselines in collision rate and planning L2, across all computational budgets.
![](figures/viz.png)
Qualitative results: In Figure 5, we show examples of our learned attention compared to baselines. As expected, our model focuses on the road and vehicles directly ahead; however, it also diverts some amount of attention to distant vehicles and road markings. This ability to dynamically distribute attention is likely why our model outperforms the baselines, which attend either indiscriminately (Dense and Road) or too selectively (Vehicle and Proximity). From the visualizations, we can better understand our model’s improved collision avoidance. Since the attention is dynamic, our model is more effective at anticipating other vehicles resulting in more cautious planning. This is illustrated by Columns A, B in Fig. 5, where our planned trajectory avoids future collisions with others. The failure cases mostly arise from rear-end collisions, one of which is Column D where all models are hit by the trailing vehicle. Note that this arises as we evaluate in open-loop. Our model focuses on surrounding vehicles and not enough on the open road to its right, which would give the option of making a right turn.
Sparsity of learned attention: Table 2 shows the result of varying from Eq. 9, which weights the regularization term from Eq. 17, with other settings held constant. We found that overall motion planning performance improves with increased sparsity, or essentially more focused computation, and peaks at 95% sparsity.
Sparsity | Planning L2 | Collision Rate | Lane Violation | |
---|---|---|---|---|
at 3s (m) | over 3s (%) | over 3s (%) | ||
Dense 0% | - | 2.207 | 0.639 | 1.350 |
Ours 20% | 2.179 | 0.547 | 1.352 | |
Ours 50% | 2.138 | 0.620 | 1.361 | |
Ours 75% | 2.132 | 0.584 | 1.387 | |
Ours 95% | 2.102 | 0.511 | 1.338 | |
Ours 99% | 2.211 | 0.566 | 1.367 |
Model | Planning L2 | Collision Rate | Lane Violation |
---|---|---|---|
at 3s (m) | over 3s (%) | over 3s (%) | |
2.210 | 0.548 | 1.378 | |
2.102 | 0.511 | 1.338 | |
2.230 | 0.621 | 1.393 | |
2.194 | 0.637 | 1.405 | |
2.188 | 0.633 | 1.397 |
Perception and prediction (PnP) loss reweighting: Table 3 shows results with varying and from Eq. 16 which control the weighting of the PnP loss computed on actors inside vs. outside the attention mask. All other variables are fixed, including sparsity at 95%. As increases, the learned attention is less restricted by detection performance on all actors, and is able to focus on only the most important actors and parts of the road, distributing attention towards improving motion planning performance. Note that is an extreme case where PnP loss is computed only on actors within attention mask: the model learns to cheat by generating attention that avoids all actors resulting in no PnP learning signal, hence the poor performance. For our main experiments, we fix .
Detection performance: Since the overall goal is improved motion-planning with lighter computation, focusing on accurately detecting all actors indiscriminately would contradict the purpose of our learned sparse attention. We should not care as much about far away or irrelevant actors that have no effect on safe planning, and should instead focus our computation on important input regions. Table 4 compares detection performance between our learned attention and the baseline dense model evaluated on different subsets of actors in the scene. Full includes all actors in the input, while Attended Region is the subset of actors that lie within the attention mask. For evaluating the dense model, we use the attention mask generated by our learned model to get the Attended Region, ensuring that both models are evaluated on the same actor subsets in both settings. The results shows that our 95% sparse, learned attention model is better than the dense model at detecting actors within the attention mask, meaning that its performance is better focused on actors that it believes are important. This may explain the overall improved planning performance of our attention-driven models as demonstrated in the main quantitative and qualitative results.
Model | mAP on Full | mAP on Attended Region | ||||
---|---|---|---|---|---|---|
IoU@0.3 | IoU@0.5 | IoU@0.7 | IoU@0.3 | IoU@0.5 | IoU@0.7 | |
Dense SA-NMP | 97.8 | 94.7 | 80.3 | 94.1 | 93.3 | 87.9 |
Ours 95% Sparse | 96.3 | 92.1 | 74.9 | 94.2 | 93.8 | 88.5 |
5 Conclusion
In this work we propose an end-to-end learned, sparse visual attention mechanism for self-driving, where the sparse attention mask gates the feature backbone computation. As opposed to existing methods that focus on using attention for perception only, our attention masks are directly optimized for motion planning, which enables our network to output better planned trajectories while achieving more efficiency with higher sparsity. As future work, the attention module can be extended to have recurrent feedbacks from the output layers to better leverage temporal information.
References
- [1] (2015) Multiple object recognition with visual attention. In ICLR, Cited by: §2.
- [2] (2015) Neural machine translation by jointly learning to align and translate. In ICLR, Cited by: §2.
- [3] (2013) Estimating or propagating gradients through stochastic neurons for conditional computation. CoRR abs/1308.3432. Cited by: §3.2.
- [4] (2019) NuScenes: A multimodal dataset for autonomous driving. CoRR abs/1903.11027. Cited by: §4.
- [5] (2018) GradNorm: gradient normalization for adaptive loss balancing in deep multitask networks. In ICML, Cited by: §2.
- [6] (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT, Cited by: §2.
- [7] (2018) Baidu apollo EM motion planner. CoRR abs/1807.08048. Cited by: §2.
- [8] (2017) Spatially adaptive computation time for residual networks. In CVPR, Cited by: §1, §2.
- [9] (2016) PerforatedCNNs: acceleration through elimination of redundant convolutions. In NIPS, Cited by: §2.
- [10] (2018) 3D semantic segmentation with submanifold sparse convolutional networks. In CVPR, Cited by: §2.
- [11] (2019) Progressive sparse local attention for video object detection. In ICCV, Cited by: §1.
- [12] (2018) Dynamic task prioritization for multitask learning. In ECCV, Cited by: §2.
- [13] (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20 (11), pp. 1254–1259. External Links: ISSN 0162-8828 Cited by: §2.
- [14] (2005) Neurobiology of attention. Academic Press, Burlington. Cited by: §2.
- [15] (2017) Categorical reparameterization with gumbel-softmax. In ICLR, Cited by: §3.2.
- [16] (2017) Cnvlutin2: ineffectual-activation-and-weight-free deep neural network computing. CoRR abs/1705.00125. Cited by: §2.
- [17] (2009) Learning to predict where humans look. In ICCV, Cited by: §2.
- [18] (2018) Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In CVPR, Cited by: §2.
- [19] (2015) Adam: A method for stochastic optimization. In ICLR, Cited by: §4.1.
- [20] (2019) Pixel-wise attentional gating for scene parsing. In WACV, Cited by: §2.
- [21] (2019) Risk-sensitive generative adversarial imitation learning. In AISTATS, Cited by: §2.
- [22] (2010) Learning to combine foveal glimpses with a third-order boltzmann machine. In NIPS, Cited by: §2.
- [23] (2017) Not all pixels are equal: difficulty-aware semantic segmentation via deep layer cascade. In CVPR, Cited by: §2.
- [24] (2020) PnPNet: end-to-end perception and prediction with tracking in the loop. In CVPR, Cited by: §3.2, §3.3, §4.
- [25] (2019) Adaptive auxiliary task weighting for reinforcement learning. In NeurIPS, Cited by: §2.
- [26] (2015) Sparse convolutional neural networks. In CVPR, Cited by: §2.
- [27] (2019) End-to-end multi-task learning with attention. In CVPR, Cited by: §2.
- [28] (2017) Learning efficient convolutional networks through network slimming. In ICCV, Cited by: §2.
- [29] (2016) Hierarchical question-image co-attention for visual question answering. In NIPS, Cited by: §2.
- [30] (2017) The concrete distribution: A continuous relaxation of discrete random variables. In ICLR, Cited by: §3.2.
- [31] (1997) Sparse coding with an overcomplete basis set: a strategy employed by v1?. Vision Research 37, pp. 3311–3325. Cited by: §2.
- [32] (2018) Tackling occlusions & limited sensor range with set-based safety verification. In ITSC, Cited by: §2.
- [33] (2018) Computationally efficient fail-safe trajectory planning for self-driving vehicles using convex optimization. In ITSC, Cited by: §2.
- [34] (2019) Agent prioritization for autonomous navigation. In IROS, Cited by: 2nd item, §2.
- [35] (2018) SBNet: sparse blocks network for fast inference. In CVPR, Cited by: §1, §2, §3.2, §3.2, §3.2, §4.2.
- [36] (2017) End-to-end instance segmentation with recurrent attention. In CVPR, Cited by: §1, §2.
- [37] (2018) Learning to reweight examples for robust deep learning. In ICML, Cited by: §2.
- [38] (2015) U-Net: convolutional networks for biomedical image segmentation. In MICCAI, Cited by: §3.2.
- [39] (2020) Perceive, predict, and plan: safe motion planning through interpretable semantic representations. In ECCV, Cited by: §2.
- [40] (2019) Jointly learnable behavior and trajectory planning for self-driving vehicles. In IROS, Cited by: §2.
- [41] (2017) On a formal model of safe and scalable self-driving cars. CoRR abs/1708.06374. Cited by: §2.
- [42] (2017) Speeding up convolutional neural networks by exploiting the sparsity of rectifier units. CoRR abs/1704.07724. Cited by: §2.
- [43] (1990) Path generation for robot vehicles using composite clothoid segments. Cited by: §3.1.
- [44] (2018) Risk-sensitive inverse reinforcement learning via semi- and non-parametric methods. I. J. Robotics Res. 37 (13-14). Cited by: §2.
- [45] (2018) Decision- time postponing motion planning for combinatorial uncertain maneuvering. In ITSC, Cited by: §2.
- [46] (2017) Attention is all you need. In NIPS, Cited by: §2.
- [47] (2017) Residual attention network for image classification. In CVPR, Cited by: §1, §2, §3.2.
- [48] (2020) Testing the safety of self-driving vehicles by simulating perception and prediction. In ECCV, Cited by: §2.
- [49] (2015) Show, attend and tell: neural image caption generation with visual attention. In ICML, Cited by: §2.
- [50] (2016) Stacked attention networks for image question answering. In CVPR, Cited by: §2.
- [51] (2019) End-to-end interpretable neural motion planner. In CVPR, Cited by: Figure 2, §2, §3.1, §3.1, §3.3, Table 1.
- [52] (2020) DSDNet: deep structured self-driving network. In ECCV, Cited by: §2.
- [53] (2017) Pyramid scene parsing network. CVPR. Cited by: §4.1.