Add support for dynamic control flow of auto placement. #476
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
修复之前auto placement不支持dynamic control flow的问题。
目前仍然有以下几个点需要注意或以后解决:
ones
,zeros
的调用,但oneflow 0.9.0对它们的实现是调用了Ones().__call__()
,这样会导致fx自动对__call__
进行包装从而使其出错。目前oneflow master分支已经修改成了forward,所以onefx暂时没有做处理。目前要跑通需要用到oneflow的master分支。compile_auto_placement
函数改成返回一个interpreter,在外面则是接收返回值之后调用run
。之前直接返回GraphModule的方式不行,这可能是一个BUG,需要以后修复。generate
函数,这样是不成立的,因为fx就是针对Module的forward进行trace的。这个和2无关,不论是否使用interpreter都是这样。这个可能可以实现,但是肯定会需要自己指定要trace的函数名,这部分后续需要再讨论一下。compile_auto_placement
的时候,不用指定concrere_args,但是有一个约定就是如果模型实际运行的时候会指定某个参数,那就需要对这个参数以keyword arg的形式传入compile_auto_placement
。这个方式后续可以讨论修改一下。