TODO.md

C++ implementation

• check the simulation implementation.
• complete algorithm.
• add Mujoco and Envs.
• DB class for training.
• Trainig method in (controller/separate class).
• link testing with github to perform auto testing.

Testing

• Testing software.
• verify implementation.
  • cost.
  • model.
  • main_algo.
  • decompose update function for simpler testing.
  • test shif and init see also #3
• Write argument checks and test them.

Performance

• profile the graph with Profiler tool
• Identify bottleneck and see how to improve performances.

Code clearness

• rewrite env class.

Documentation

• Write documentation.
  • Model base.
  • Cost base.
  • Controller base.
  • auv_model.
  • point mass model.
  • nn model.
  • static cost.
  • elipse cost.
• Doxygene?

Python Implementation

implementations:

• add elements to replay buffer.
• finish training method.
• Rewrite Testing.
• log cost. (~1/2 day)
• extend dimensions. (~1/2 day)
• test extended dimension. (~1 day)
• generate tf_summary with hyperparam and env name. (~2 days)
  • add config file parsing, to load hyperparameters and remove hard coding of the params.
  • add argument parsing for the config file, env file etc. This will be usefull for organised testing.
• sort rendering problem.
• plot weighted simulated paths.
• look at noise distribution.
• log weights.
• log nabla in % of K.
• change the goal be a signal.
• look at signal filtering for control input.
• decoupling temprature from control cost.
• implement covariance decoupling between natural noise and artifical noise. Simple variance multiplier first.
• save task_config file.
• log inputs.
• log best cost.
• Sort tensorflow memory usage.
• normailze cost
• log normalized cost.
• debug flag when logging summary.
• log predicted cost.
• train only if enough samples.
• moved logging to a separated pseudo singleton.
• log speed and position cost for elipse.
• log arguments in the graphs dir to repeate experiments.
• write experiment bash script.
• write repeate experience from log script.
• write model abstract class.
• write documentation.
• perform argument checking.
• tune filtering process.
• log disturbances.
• add current. Needs to be parameterizable.
• Test Cpu. (probably inference on GPU, learning on CPU)
• Train simple network on Point mass system.
• Complexify learning (NN, GP etc). (~1-2 weeks) need to implement model, search the hyperparameters, same if we try Gaussian processes (GPflow) probably not a priority.
• Experiment agent learning parameters. (dont know enough about the rest to make a prediction)

NN_AUV implementation:

Main Idea: We don't want to predict in the inertial frame because this would lead to a lack of generalization when far from the origin in unexplored state space regions. So the dataset is as follos: A state is defined by: [pose_I_Bt.T, vel_Bt_Bt.T].T where pose_I_Bt is the pose of the body at time t expressed in inertial frame and vel_Bt_Bt is the velocity of the body frame as time t expressed in the body frame at time t. For 2 states, s_{t}, s_{t+1} as well as the input action a_{t} the network training data is as follows: (X, y) \simeq ([rot_I_Bt.T, vel_Bt_Bt.T, a_Bt_Bt.T].T, [pose_Bt_Bt1.t, vel_Bt1_Bt1.T].T). Where pose_Bt_Bt1 is the pose of the body at time t+1 expressin in the body frame at time t.

• Batch transform.
• Batch invsere.
• Batch quaternion inverse.
• Batch pure quaternion.
• Batch rotate vector.
• Batch quaternion multiplication.
• Data Preparation.
• Training Data Preparation
• Prediction to Inertial Frame.
• Create network.
• Training routine.
• Collect training data from gazebo.
• Train simple network on the training data.

ROS Integration

• run a tf-graph in ros with python3.

• adapt static cost and eliptic cost to 3d problems.

• vehicle modelling.

  • step function.
  • compute acceleration.
  • compute damping.
  • compute restoring forces.
  • compute coriolis matrix.
  • load ros model parameters.
    • namespace.
    • world frame.
    • body frame.
    • inertial.
    • mass.
    • volume.
    • density.
    • height.
    • length.
    • width.
    • center of gravity.
    • center of buoyancy.
    • added mass.
    • gravity?
    • linear damping.
    • linear damping forward speed.
    • quadratic damping.
    • build mass matrix
  • test step function.
  • Why is model exploding (sign error in the restoring forces).
    • Damping matrix computation had an error. Done, compared with the UUV implementation and have no diff anymore.
    • Rerun the data gen process and see if it still explodes. It still explodes. Need to look at the angles as they are not normalized.
    • Implement quaternions.
    • Test quaternions.
      • check rot.
      • check t.
      • verify order of multiplication.
      • check jacobian.
      • check multiplication in step.
      • check converstion.
  • look for learning mechanism.
  • write test functions.
  • change variables to tensorflow variables.
  • find solution to skew operation concat .

• Performance testing.

• go to real robot.

Troubleshooting:

• check why the acceleration in x is positive while the yaw angle is 180 and the input is 100N. The x acceleration should be negative.

Testing

• Test Model.
• Test Cost.
• Test main.
• Test shift and init.
• Test elips cost.
• Test gamma and upsilon.