feat: plaintext support #1499
feat: plaintext support #1499
Conversation
Can now export plaintext python files in the `percent`, `light`, `sphinx`, `myst` and `pandoc` formats. The new feature uses `jupytext` to convert the `.{pct,lgt,spx}.py` and `.{myst,pandoc}.md` files to an intermediate `.ipynb` file, which is then processed by `nbdev` in the usual manner.
To export a plaintext file (in the `percent` format):
```python
nbdev.export.nb_export("my_file.pct.py", lib_path="my_lib")
```
To export plaintext files in the `nbs` folder (in the `percent` format):
```bash
nbdev_export --file_glob '*.pct.py'
```
Example plaintext files are in `tests/plaintext_files`. We run through the exporting process of each of these files into an `.ipynb` notebook.
…port The `fmt` argument takes any of the following: 'py:percent', 'py:light', 'py:sphinx', 'md:myst', 'md:pandoc' and 'ipynb'.
`nbdev_export` now exports files of the form "*.{pct.py,lgt.py,spx.py,myst.md,pandoc.md,ipynb}" by default.
In order to do this, I added support for multiple file globs in `nbglob`, and adjusted the default file glob in `nbglob_cli` to be for all support plaintext and notebook filetypes.
|
Note: In this commit I change the default behaviour of nbdev_export --file_glob '*.pct.py' |
|
Thanks for providing the blog post with the motivation for this PR. Plain text formats don't capture outputs which are required to render nbdev docs so I'm afraid this is not a good fit. Please let me know if I'm wrong or I am overlooking something! |
|
Hi @hamelsmu. Thanks for taking the time to look through the PR and issue. I agree that plaintext formats do not capture outputs, but I don't think these are required as such to render them via Quarto, which itself has support for rendering notebook-formatted I take your point that we often want to render the outputs in the docs as well, although for some use-cases (like mine) it's less necessary. One option would be to add a pre-processing step where the plaintext files are converted to |
|
I have used In my opinion, there is a great benefit on not having to deal with the outputs under version control, particularly when they are binary objects (like images). Besides the pain to have to use a paid tool for proper code reviews, changes in binary files increase the size of the git history massively, making it quickly become annoying to deal with |
|
PS Quarto also has the option to execute cells when rendering: https://quarto.org/docs/computations/execution-options.html |
There was a problem hiding this comment.
Thank you for this PR. Let's make this opt in, disabled by default. You can add a setting for it. I've added some minor formatting requests.
It's unusual to have a separate "tests" notebook. Can this be integrated into the docs instead, creating tests that also illustrate the behavior?
| res = globtastic(path, file_glob=file_glob, skip_folder_re=skip_folder_re, | ||
| skip_file_re=skip_file_re, recursive=recursive, **kwargs) | ||
| if type(file_glob) != list: file_glob = [file_glob] | ||
| res = [] |
There was a problem hiding this comment.
This can be a list comprehension.
| @@ -138,7 +142,7 @@ def nbglob_cli( | |||
| @call_parse | |||
| @delegates(nbglob_cli) | |||
| def nbdev_export( | |||
| path:str=None, # Path or filename | |||
| path:str=None, # Path or filename, | |||
| @@ -16,6 +16,14 @@ | |||
|
|
|||
| from collections import defaultdict | |||
|
|
|||
| try: | |||
| import nbformat | |||
| except ImportError: | ||
| plaintext_supported = False |
There was a problem hiding this comment.
| except ImportError: | |
| plaintext_supported = False | |
| except ImportError: plaintext_supported = False |
| @@ -88,17 +96,43 @@ def _mk_procs(procs, nb): return L(procs).map(instantiate, nb=nb) | |||
| # %% ../nbs/api/03_process.ipynb | |||
| def _is_direc(f): return getattr(f, '__name__', '-')[-1]=='_' | |||
|
|
|||
| # %% ../nbs/api/03_process.ipynb | |||
| plaintext_file_formats = { | |||
| nbformat.write(nb_converted, temp_file.name) | ||
| self.nb = read_nb(temp_file.name) if nb is None else nb | ||
| return | ||
| if fmt is None or fmt == "ipynb": |
There was a problem hiding this comment.
Each part here can be on one line.
|
Thanks @jph00 for your comments.
Sounds good, will do!
Sure, can integrate these into the docs. |
Closes #1498
Adds support for exporting the following plaintext formats as if they were regular
.ipynbfiles:percentlightsphinxmystpandocSee this for details on the above plain-text notebook formats.
Running
nbdev_exportnow converts, by default, files of extension*.{pct.py,lgt.py,spx.py,myst.md,pandoc.md,ipynb}. See this for a demonstration (and some documentation) on the new exporter.The feature requires
jupytextandnbformatas (optional) dependencies. They will only be required if users attempt to export non-ipynb files.