-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
24 additions
and
6 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,31 @@ | ||
# actaim2-eccv24 | ||
# Modes of interaction learning for unsupervised affordance discovery | ||
|
||
Test editing website | ||
## Problem Statement | ||
|
||
Test putting on png image | ||
In the realm of object manipulation, human engagement typically manifests through a constrained array of discrete maneuvers. This interaction can often characterized by a handful of low-dimensional latent actions, such as the act of opening and closing a drawer. Notice that such interaction could diverge on different types of objects but the interaction mode such as opening and closing is discrete. In this paper, we explore how the learned prior emulates this limited repertoire of interactions and if such a prior can be learned from unsupervised play-data. we take a perspective that decomposes the policy into two distinct components: a skill selector and a low-level action predictor, where the skill selector is operating within a discretely structured latent space. | ||
|
||
![cake_halfcut](https://github.com/pairlab/actaim2-eccv24/assets/30140814/b72aa716-ae45-4412-86b4-a431641deda8) | ||
We introduce ActAIM2, which given an RGBD image of an articulated object and a robot, identifies meaningful interaction modes like opening drawer and closing drawer. ActAIM2 represents the interaction modes as discrete clusters of embedding. ActAIM2 then trains a policy that takes cluster embedding as input and produces control actions for the corresponding interactions. | ||
|
||
Test putting on video | ||
<img width="1238" alt="teaser_3" src="https://github.com/pairlab/actaim2-eccv24/assets/30140814/687daaa0-3cb3-4697-b3f8-b33d5351b7dd"> | ||
|
||
|
||
https://github.com/pairlab/actaim2-eccv24/assets/30140814/fecc9fac-3095-4325-be59-ce9a35c0e0d0 | ||
|
||
### sample object data collection | ||
|
||
run | ||
|
||
``` | ||
. collect_data.sh | ||
``` | ||
|
||
to collect dataset. | ||
|
||
### To access full object dataset from where2act | ||
|
||
## Training | ||
|
||
run | ||
|
||
```bash | ||
python scripts/train_aff.py | ||
``` |