Program documentation: md vs dox

[jgvictores]

Program documentation:

1. dox : most repos
2. md : look at roboticslab-uc3m/tools@89578f3

[PeterBowman]

Do you by chance mean that those huge chunks of how-to-use comments like this (usually located in the app's main.cpp) should be moved to a .md file?

[jgvictores]

I refer to that type of movement, but rather than a suggestion, it is a question.

IMHO, the pros are:

1. dox: autogeneration of link pages (avoiding hard-coded to master/develop like here).
2. md: online rendering, and perhaps more visibility in general.

What is your opinion? Maybe there is an option to link md via relative paths?

[RaulFdzbis]

Personally I prefer md because of these reasons:

1. It's integrated in github (This give md. more visibility and familiar use, you are using github all the time, dont need to change between github-doxygen)
2. In my personal experience most of the repos i have found have most of it's documentation in md.
3. I think maybe doxygen is too program oriented, so its harder to read for those people that are not familiar with programming.

On the other hand, it has some cons:

1. Can get messy if the documentation is too big (i don't think right now is a problem for us, also there are better options for big documentations like gitbook)
2. Harder to maintain, at the end md uses something similar to absolute paths.
3. We use doxygen in most of our repos.

[David-Estevez]

My personal opinion: I only like Doxygen for documenting APIs, classes, etc. For any other doc info I prefer markdown because of the simplexity of the syntax yet beautiful syntax. :)

[PeterBowman]

Sorry for my late response. I'm getting increasingly convinced that MD might be a nice replacement for long, unwieldy Doxygen doc files. We have a bunch of them scattered around asibot-main (see doc.dox). The syntax is rather obscure and lacks GitHub editor support (which is essentially a WYSIWYG markdown text editor).

dox: autogeneration of link pages (avoiding hard-coded to master/develop like here).

@jgvictores I think that relpaths could work there, try programs/openraveToStl (or /programs/openraveToStl, or ./programs/openraveToStl).

[PeterBowman]

Another remark: developers should feel impelled to keep their app documentation synced with any related change they commit. Doc files buried far from the source code or even a cumbersome syntax (with lack of editor support, as noted before) may lead to a situation in which devs either simply forget about the existence of related documentation or feel uncomfortable to edit those few lines. A clean workflow, stating what needs to be done before commiting (reflect changes in docs, update release notes, etc.), should follow from these considerations. Outdated documentation is evil. Compare YARP's CONTRIBUTING.md.

[jgvictores]

@jgvictores I think that relpaths could work there, try programs/openraveToStl (or /programs/openraveToStl, or ./programs/openraveToStl).

@PeterBowman Correct, fixed at roboticslab-uc3m/tools@567a8f7

[jgvictores]

I believe we haven't provided options on how to perform this migration. Going to check out: https://www.npmjs.com/package/doxygen2md

[RaulFdzbis]

I believe we haven't provided options on how to perform this migration. Going to check out: https://www.npmjs.com/package/doxygen2md

Looks good!, at least to deserve a try!

[jgvictores]

https://www.npmjs.com/package/doxygen2md

Bad thing is it's not direct like Pandoc. Steps are using Doxygen to create an XML, then apply it. Alternatives I've seen are similar. Let's keep looking.

[jgvictores]

Refs of different ways we do and link things:

• https://github.com/roboticslab-uc3m/vision specifically link commented above
• Possibly https://github.com/roboticslab-uc3m/tools
• For sure https://github.com/roboticslab-uc3m/yarp-devices (https://github.com/roboticslab-uc3m/yarp-devices/blob/develop/doc/yarp-devices-usage.md)
• Definitively https://github.com/roboticslab-uc3m/speech see result of Document ASR and TTS speech#10

[jgvictores]

Would be nice to be able to link, e.g:

• http://robots.uc3m.es/dox-openrave-yarp-plugins/classroboticslab_1_1YarpOpenraveRGBDSensor.html
• https://github.com/roboticslab-uc3m/openrave-yarp-plugins/tree/develop/libraries/YarpPlugins/YarpOpenraveRGBDSensor

Cannot find a quick and easy way to do this.

[jgvictores]

To sum up: we prefer .md for most situations, except existing .dox which just set up entry point and groups.

• Tracking: Migrate doxygen documentation that is almost hidden within the code to README.md per-folder.
• Do once: Set Doxygen up to generated documentation to link to GitHub URL of new README.md files.

[PeterBowman]

Do once: Set Doxygen up to generated documentation to link to GitHub URL of new README.md files.

I've set up a dox target in project-generator and install rules that also take care of proper packaging. That is, you run make dox and make package, then CMake and CPack generate HTML docs and pack them for proper distribution. We can't achieve this if relying on URLs. Proposal: convert Markdown files (which are suitable for GH and online viewing) into an exportable markup and generate convenient install and packaging rules. Might be a long-term task.

[jgvictores]

Oh. Something like pandoc to locally convert from .md to .html could be interesting. 👍

[PeterBowman]

Note: Doxygen supports Markdown! (reference)

Markdown support was introduced in doxygen version 1.8.0. It is a plain text formatting syntax written by John Gruber, with the following underlying design goal:

Also: Doxygen allows tranclusion (\include command), while Markdown doesn't.

Edit: only partial support, i.e. links to local raw HTML files or HTMLs served remotely (full URLs). No transclusion.

[jgvictores]

Since \includedoc isn't parsed on time, very tempted to pre-process as in https://stackoverflow.com/questions/7857726/sed-replace-backslash-in-include

[jgvictores]

https://github.com/Anvil/bash-doxygen

[jgvictores]

https://stackoverflow.com/questions/9502426/how-to-make-an-introduction-page-with-doxygen

[PeterBowman]

Idea: pass a path to a script in INPUT_FILTER or FILTER_PATTERNS (Doxyfile options) in order to replace a placeholder text inlined in .h docs with the actual MD file contents. Problem (see bolded text):

The INPUT_FILTER tag can be used to specify a program that doxygen should invoke to filter for each input file. Doxygen will invoke the filter program by executing (via popen()) the command:
<filter> <input-file>
where <filter> is the value of the INPUT_FILTER tag, and <input-file> is the name of an input file. Doxygen will then use the output that the filter program writes to standard output. If FILTER_PATTERNS is specified, this tag will be ignored.
Note that the filter must not add or remove lines; it is applied before the code is scanned, but not when the output code is generated. If lines are added or removed, the anchors will not be placed correctly.
Note that for custom extensions or not directly supported extensions you also need to set EXTENSION_MAPPING for the extension otherwise the files are not properly processed by doxygen.

Reference: http://doxygen.nl/manual/config.html#cfg_input_filter.

[jgvictores]

Yes, I'm also concerned about those bold words.

[jgvictores]

Simple commands such as INPUT_FILTER = "sed s/FROM/TO/" work.

Maybe awk would be better than sed.

Before I forget: http://doxygen.10944.n7.nabble.com/input-filter-woes-td2056.html

[jgvictores]

Also interesting: https://stackoverflow.com/questions/19704822/how-to-use-markdown-pages-and-doxygen-groups

[jgvictores]

It looks like in general sed and \n are not big friends: https://askubuntu.com/questions/603096/replace-string-with-multiline-file-content

[jgvictores]

Just found a tool similar to Doxygen (can't tell the difference): https://github.com/copperspice/doxypress

[jgvictores]

https://news.ycombinator.com/item?id=10165707

[PeterBowman]

There is a nice intermix of Markdown and Doxygen capabilities in libraries/YarpCloudUtils/README.md@vision. Even though the document has been written in Markdown, Doxygen is perfectly capable of coping well with it: doxy page. Note that documented classes and functions are automatically linked by Doxygen in text and code snippets. It is also trivial to link back to this page from usual doxy lines, see note.

[PeterBowman]

YARP is moving towards a deeper integration between Markdown and Doxygen. This patchset esentially migrates a .dox into an .md and adds several Doxygen-like commands (\snippet, \subpage, \ref) throughout those .md files to perform transclusion at the Doxygen level. The documentation is not fully functional on the Markdown side, yet it seems to combine the best of the two worlds.