Zarifikram

.gitignore

@@ -1,4 +1,9 @@
 .idea/
-data/
 *.csv
 *.pyc
+*.mp4
+*.dill
+*.dll
+*.zip
+data/ind
+*.ai

.vscode/settings.json

@@ -0,0 +1,3 @@
+{
+    "python.pythonPath": "C:\\Users\\adhocmaster\\anaconda3\\envs\\drone-dataset-tools38\\python.exe"
+}

README.md

@@ -1,9 +1,14 @@
 # Drone-Dataset Tools
 
+
 The goal of this repository is to make using [drone datasets](https://www.drone-dataset.com/) as easy as possible. 
 Therefore, we provide source code in Python for import and visualization. 
 Thus, this source code not only allows to visualize trajectories and thus get an overview, but also serves as a template for your own projects.
 
+## Extractor documentation:
+Read the documentation [here](./doc/extractors.md) for the extractor API.
+Read the documentation [here](./doc/visualization.md) for the visualization API..
+
 ## Installation and Quick Start
 
 1. Create a new Python environment or select a pre-existing one. 
@@ -90,6 +95,7 @@ All available options are listed in the following table:
 | `--annotate_orientation`    | `False`           | Annotate every track by its current orientation. | 
 | `--annotate_age`            | `False`           | Annotate every track by its current age. | 
 | `--show_maximized`          | `False`           | Show the track Visualizer maximized. Might affect performance. | 
+| `--ped_only`                | `False`           | Only render frames with pedestrians and a button to jump to the next pedestrian's initial frame. | 
 
 *Please note that drawing additional features may decrease the playback animation update rate.*
 

data/scenes/ind.json

@@ -0,0 +1,194 @@
+{
+  "2": {
+
+    "210": {
+      "centerXPixel": 628.5,
+      "centerYPixel": 286.5,
+      "centerX": 61.44,
+      "centerY": -28.01,
+      "angle": 12.17,
+      "roadWidth": 5.0,
+      "boxWidth": 10.0
+    },
+
+    "211": {
+      "centerXPixel": 688.5,
+      "centerYPixel": 273.5,
+      "centerX": 67.30,
+      "centerY": -26.74,
+      "angle": 12.17,
+      "roadWidth": 5.0,
+      "boxWidth": 5.0
+    },
+
+    "220": {
+      "centerXPixel": 383.5,
+      "centerYPixel": 322.5,
+      "centerX": 37.49,
+      "centerY": -31.53,
+      "angle": 16.0,
+      "roadWidth": 6.0,
+      "boxWidth": 15.0
+    },
+
+    "221": {
+      "centerXPixel": 302.5,
+      "centerYPixel": 345.5,
+      "centerX": 29.57,
+      "centerY": -33.77,
+      "angle": 16.0,
+      "roadWidth": 9.0,
+      "boxWidth": 15.0
+    },
+
+    "222": {
+      "centerXPixel": 188.5,
+      "centerYPixel": 375.5,
+      "centerX": 18.43,
+      "centerY": -36.71,
+      "angle": 16.0,
+      "roadWidth": 9.0,
+      "boxWidth": 20.0
+    },
+
+    "230": {
+      "centerXPixel": 490.5,
+      "centerYPixel": 392.5,
+      "centerX": 47.95,
+      "centerY": -38.36,
+      "angle": 111.79,
+      "roadWidth": 6.0,
+      "boxWidth": 15.0
+    },
+
+    "231": {
+      "centerXPixel": 518.5,
+      "centerYPixel": 461.5,
+      "centerX": 50.67,
+      "centerY": -45.11,
+      "angle": 111.79,
+      "roadWidth": 6.0,
+      "boxWidth": 10.0
+    },
+
+    "240": {
+      "centerXPixel": 450.5,
+      "centerYPixel": 180.5,
+      "centerX": 44.04,
+      "centerY": -17.64,
+      "angle": 134.74,
+      "roadWidth": 6.0,
+      "boxWidth": 10.0
+    }
+
+  },
+
+ "4":{
+
+  "440": {
+    "centerXPixel": 700,
+    "centerYPixel": 499,
+    "centerX": 106.68,
+    "centerY": -76.05,
+    "angle": 26.11,
+    "roadWidth": 18.0,
+    "boxWidth": 50.0
+  },
+
+  "441": {
+    "centerXPixel": 1085,
+    "centerYPixel": 315,
+    "centerX": 165.35,
+    "centerY": -48.01,
+    "angle": 23.74,
+    "roadWidth": 18.0,
+    "boxWidth": 25.0
+  },
+
+  "430": {
+    "centerXPixel": 908.28,
+    "centerYPixel": 240.47,
+    "centerX": 138.42,
+    "centerY": -36.65,
+    "angle": 84.42,
+    "roadWidth": 18.0,
+    "boxWidth": 40.0
+  }
+  },
+
+  "1":{
+
+   "140": {
+     "centerXPixel": 663.27,
+     "centerYPixel": 230.52,
+     "centerX": 64.84,
+     "centerY": -22.53,
+     "angle": 49.09,
+     "roadWidth": 6.0,
+     "boxWidth": 15.0
+   },
+
+   "130": {
+     "centerXPixel": 716.38,
+     "centerYPixel": 498.85,
+     "centerX": 70.03068040678744, 
+     "centerY": -48.765745722836925,
+     "angle": 132.18,
+     "roadWidth": 6.0,
+     "boxWidth": 30.0
+   },
+
+   "131": {
+     "centerXPixel": 414.91,
+     "centerYPixel": 166.13,
+     "centerX": 40.56007929811019,
+     "centerY": -16.240259270191235, 
+     "angle": 132.18,
+     "roadWidth": 6.0,
+     "boxWidth": 30.0
+   },
+
+   "120": {
+     "centerXPixel": 522.15,
+     "centerYPixel": 400.08,
+     "centerX": 51.043468235299784,  
+     "centerY": -39.11035290927653,
+     "angle": 45,
+     "roadWidth": 6.0,
+     "boxWidth": 10.0
+   }
+   },
+
+  "3":{
+
+   "340": {
+     "centerXPixel": 596.30,
+     "centerYPixel": 523.31,
+     "centerX": 58.292100179468086,
+     "centerY": -51.156865579267894,
+     "angle": 51,
+     "roadWidth": 20.0,
+     "boxWidth": 6.0
+   },
+
+   "330": {
+     "centerXPixel": 525.85,
+     "centerYPixel": 207.48,
+     "centerX": 51.40516666002566, 
+     "centerY": -20.28248355733027,
+     "angle": 14.84,
+     "roadWidth": 6.0,
+     "boxWidth": 6.0
+   },
+
+   "331": {
+     "centerXPixel": 583.68,
+     "centerYPixel": 267.57,
+     "centerX": 57.05841528215987, 
+     "centerY": -26.156661487540294, 
+     "angle": 82.85,
+     "roadWidth": 2.0,
+     "boxWidth": 6.0
+   }
+   }
+}

data/scenes/ind_crazy_pedestrians.json

@@ -0,0 +1,3 @@
+{
+    "19": [19, 40, 47, 49,  75, 125, 150, 178, 232, 242, 282, 283, 288, 302, 341, 343]
+}

data/scenes/ind_invalid_pedestrians.json

@@ -0,0 +1,3 @@
+{
+    "19": [41, 63]
+}

doc/README.md

@@ -0,0 +1,78 @@
+# Pedestrian trajectory extraction and processing
+
+This tool provides extraction and processing scripts for jaywalker trajectories. It currently extracts data from InD dataset. But the approach can be used for any BEV dataset.
+
+## Architecture & Tutorials
+1. [Methodology for Extraction](extractors-methodology.md) 
+2. [Quick start](how-to-use.md)
+3. [Extractors](extractors.md)
+4. [Visualization tools](visualization.md)
+5. [Threads to validity](threats-to-validity.md)
+
+## Goal
+
+The raw dataset has pedestrian and vehicle trajectories. The following figure shows the pedestrian trajectories. The goal of this project is to extract pedestrians that cross the road and clip their trajectories to the interesting zone only. 
+<img src="./images/location2-raw.PNG" width="600">
+
+In the following figures, the purple ones are complete trajectories and the blue ones are clipped to the crossing area of the road.
+<img src="./images/location-2-scene-10-all.PNG" width="600">
+<img src="./images/location-2-scene-21-all.PNG" width="600">
+
+
+
+## Scene coordinate system:
+<img src="./images/scene-coordinate-system.PNG" width="600">
+
+The original data has position and dynamics in the image coordinate system which is the top-left pixel of the background image. Our extracted data transforms trajectories into scene coordinate system where the origin is at the center of the scene bounding box and x axis is rotated counter-clockwise to aligh with the road reference line (length).
+
+So, our scene data looks like this (plotted in the image coordinate system)
+<img src="./images/scene-data.PNG" width="600">
+
+## Issues with the data
+The dataset is annotated by a model trained with a deep learning object tracking method. So, it sometimes fail to classify and track objects correctly. Our exploratory analysis discovered some issues with the dataset:
+
+1. Unrealistic paths - there are long pedestrian trajectories that run along the road (over 50 meters). There is a high probability that bicyles or scooters are classified as pedestrians.
+2. Unrealistic speed - Some pedestrians seems to be too fast. Interestingly, then can run as fast as 30 km/h which professional atheletes or sprintters can achieve on track or in the field. We re-labeled them as fast_pedestrian as they interact with other traffic participants.
+3. Incomplete trajectories - Pedestrians can pop up or vanish in the middle of the road. This happens when tracking fails. In the following example we have a trajectory that was both started and ended in the middle of the road. We cannot simply omit these trajectories as they interact with other trajectories.
+
+<img src="./images/incomplete-trajectory.PNG" width="600">
+
+## Interesting Patterns in midblocks
+1. Sharpe and unpredictable change in direction
+2. Lateral displacement is significant
+3. Wide variety of patterns in the paths.
+
+<img src="./images/incomplete-trajectory.PNG" width="600">
+
+## Derived Data
+
+**Additional attributes for track**
+
+| Attribute | Description |
+| --- | ------ |
+| uniqueTrackId | Unique track identifier in a location. First two digits denote the recordingId and last three digits denote the trackId in the recording. |
+| sceneId | Every scene in a location has a sceneId |
+| roadWidth | Approximate road width along the scene y-axis |
+| sceneX | track x position in the scene coordinate system |
+| sceneY | track y position in the scene coordinate system |
+
+**Scene meta data**
+
+The meta data is developed in clipped trajectories in the scene coordinate system.
+
+| Attribute | Description |
+| --- | ------ |
+| uniqueTrackId | Unique track identifier in a location. First two digits denote the recordingId and last three digits denote the trackId in the recording. |
+| initialFrame | starting frame in the recording |
+| finalFrame |  ending frame in the recording  |
+| numFrames | Life span in frames. Depends on the FPS of the data. |
+| class | type of the actor |
+| horizontalDirection | Positive x is EAST |
+| verticalDirection | Positive y is NORTH |
+
+
+## Application
+1. Trajectory inpainting or forking - helps the partial trajectories recorded in dashcams.
+2. Scenario reconstruction and dataset augmentation
+3. Generative behavior modeling
+

doc/extractors-methodology.md

@@ -0,0 +1,27 @@
+# Extracting by scene configuration
+
+We define a scene as a rectangular area on the road where interesting pedestrian crossing behavior can be found. A scene configuration has the center of the area in world coordinates, and a wdith and height. Reactangular areas have two sides running parallel to the road's center line.
+
+Overall steps:
+1. create some scene configuration (bounding boxes of scene areas)
+2. Create splines from trajectories.
+3. Keep the trajectories that overlaps with the scene bounding boxes. To do that, we only check overlap of trajectories with the middle half of the bounding boxes.
+4. Clip trajectories within 1 meter of the bounding box sides. 
+
+
+For location 2, here goes the bounding boxes. The data is available [here](../data/scenes/ind.json).
+
+<img src="./images/location-scene-annotation.png" width="600">
+
+
+# Steps
+
+```mermaid
+flowchart TD
+    LocRec[Get recordings for a location] --> Down[Downsample recording data from 25.0 fps to 2.5 fps]
+    Down -- List of RecordingData --> Scene[Get Scene trajectories for each RecordingData]
+    Scene --> div[Divide trajectories between pedestrian and non-pedestrian data]
+    div --> comb[Combine data by scene]
+    comb --> Export
+
+```