* Tasks
** DONE make this pip-installable
https://python-packaging.readthedocs.io/en/latest/command-line-scripts.html
** TODO make it work on windows
use asyncio instead of bash and tee?
** TODO raise error if conda is needed and missing
** TODO readme: show cooler examples
examples that show what you can do:

- single-file notebook server!  
- demo with axolotl
- simple webserver
- use pandas to convert CSV to excel
- use language model
- use for PDF extraction
** TODO ? readme: show example of a requirements.txt that references a dependency by URL
** TODO ? add flag --install-miniconda flag
** TODO ? consider making it installable as a brew package
** TODO develop example with test with advanced conda dependencies
Try to convert this to a single comment block:

#+begin_src sh
# create conda env
conda create -n multistep python=3.10 # higher versions might break vllm
conda activate multistep

# Linux install cuda and cuda toolkit and pytorch. Good as of 2024-05-14 Tue 15:45
conda install cuda -c nvidia/label/cuda-12.1.0
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

# conda-install conda-packaged dependencies
conda install openai networkx numpy pandas regex

# pip-install pip-only dependencies
python3 -m pip install claudette
python3 -m pip install vllm
python3 -m pip install datasets
python3 -m pip install nltk
#+end_src

** nice-to-have
** TODO ? add --force-rebuild flag
** TODO ? provide stats regarding total space used cache
** TODO ? create a custom GPT readme
** TODO ? design way to add persistent link in env to originating script
- could name directory with first script name used
  - con. hides later scripts
- could add symlink in the dir back to the script?
  - but what if different scripts have the same name? add junk to uniqiify?
** TODO ? add command line option to read Python expressions
** TODO ? implement inline (workflow for pure-venv case)
design goal:
- instead of putting this script in the path, it should also be sufficiently literally to append it to any python script
issues:
- overriding existing defintiions of main
- collisions on globals namespace
** done
** DONE update macOS cache dir based on [[https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/MacOSXDirectories/MacOSXDirectories.html#//apple_ref/doc/uid/TP40010672-CH10-SW1][apple file system programming guide]]
page defining location: [[https://developer.apple.com/library/archive/documentation/FileManagement/Conceptual/FileSystemProgrammingGuide/MacOSXDirectories/MacOSXDirectories.html#//apple_ref/doc/uid/TP40010672-CH10-SW1][macOS Library Directory Details]]

options:
- $HOME/Library/Caches/python-script/
- $HOME/Library/Developer/python-script/

* Resources

- https://rust-script.org/
- https://github.com/mxcl/swift-sh/tree/master/Examples
- scriptisto (more general, but more opaque)

* Log

** [2024-06-03 Mon 09:28] what format for conda?

Options:

1. embed environment.yml
   - works with -n if environment.yml does not specify the env name!
   - works with -prefix if environment.yml does not specify the env name!

     
3. embed whatever format =conda install --file= expects
   - that format is also what =conda install= expectsspecification?apparently yes.
   - that format is the [[https://conda.io/projects/conda/en/latest/user-guide/concepts/pkg-search.html][install specification]] 

[[https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html][conda install docs]] note that with --file conda intsll will "Read package versions from the given file."

** [2024-06-06 Thu 19:03]

packaging

https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#writing-pyproject-toml

[[https://packaging.python.org/en/latest/guides/single-sourcing-package-version/#single-sourcing-the-version][single-sourcing version]]

** [2024-06-07 Fri 11:44]

macOS Sonoma ships with Python 3.9.6 and with setuptools 58.04

setuptools can be configured with pyproject.toml only as setuptools 61.0.0

So:
- must not rely on pyproject.toml
- can use a setup.cfg

twine upload to PyPi

** [2024-06-07 Fri 14:07] two configurations:

*** config 1

#+begin_quote
.
 ├── pyproject.toml
 ├── python_script
 │  ├── __init__.py
 │  ├── __main__.py
 │  └── python_script.py
#+end_quote

properties:

- pro: $ python3 -m python_script
- pro: $ python-script
- con: then python_script.py can't just go into the path!

Conflict:
- copy-file installability is good, and requires files be named like CL tools
- pip-installability is good
- but pip requires modules to have bad names for a CL tool
  
Options:
1. duplicate the file, python-script AND python_script??
2. use =exec= to load a file without the py extension?

One might think there was an option 3, to just use a Python-compliant name also for the CLI, by calling it python_script.py or pythonscript.py. But this seems not to work. The  =python3 -m= builds a shim. When that shim runs, it does not find the package named python_script. Not clear why. In other words, there is a defect in the =python3 -m=  run mechanism such that you cannot name one of your project entry point scripts "foo.py" if the import name of your package is "foo", since then 


*** config 2?
 .
 ├── pyproject.toml
 ├── python_script
 │  ├── __init__.py
 │  ├── __main__.py
 │  └── python_script.py
 
** [2024-06-07 Fri 14:16] naming

names:
- pythonscript
- python-script
- pythonscript.py
- python_script.py

- pyrun
- pyrunscript
- python-script.py
- pyscript:
  - no used
- pydepscript
- depscript
  - no. no mention of python


** [2024-06-07 Fri 16:59] name collision

The project name "pythonscript" was rejected.

presumably because of the existing project PythonScript1, a [[https://pypi.org/project/PythonScript1/#history][dead stub of a project]]. [[https://pypi.org/project/PythonScript1/#history][collision policy]]

will try pythonrunscript

** [2024-08-20 Tue 22:32] noted

An official spec for embedded deps: https://packaging.python.org/en/latest/specifications/inline-script-metadata/#inline-script-metadata

PEP: https://peps.python.org/pep-0723/


Noted re: inline script metadata:

one example for a tool called "some-toml"

#+begin_quote
# /// some-toml
# embedded-csharp = """
# /// <summary>
# /// text
# ///
# /// </summary>
# public class MyClass { }
# """
# ///
#+end_quote

ANother example for the tool called "script":

#+begin_quote
# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "requests<3",
#   "rich",
# ]
# ///
#+end_quote

Possible use of this syntax: add support for doc comments with these tools:
- pythonrunscript-requirements.txt
- pythonrunscript-conda_install_specs.txt

current parser works by:
- defining 3 begin patterns, 1 end pattern, and 1 data pattern
- defining 4 parser patterns (3 in-block, 1 out-of-block)
- elif-based pattern matching on (line_type, match obj)

notable: https://pip.wtf/

** [2024-09-02 Mon 23:29] notes on wtf.pip

- PRO. lightweight, comprehensible. Very good. <10 LOC.
- CON. semi-nonstandard syntax for specifying deps
- CON. Only pip, no conda
- CON. installs and uses dedicated env, unconditionally, and even if current env suffices
  - inefficient.
  - interferes with other workflows, like developing the script in an IDE-managed env

Evolution?:
- only install deps if needed and confirmed?
  - q: does pip itself already providing UI prompting?
  - q: how to check deps?
    - also with pip

      
** [2024-09-06]

- [ ] Q: how to convert all version specs valid in script TOML, to specs valid in conda python specs and requirementx.txt dep specs?

- exact version format required in PEP723 TOML?

  for deps, Uses https://packaging.python.org/en/latest/specifications/dependency-specifiers/#dependency-specifiers

  for python, uses https://packaging.python.org/en/latest/specifications/version-specifiers/#version-specifiers

- exact version format required in requirements.txt?

  requirements.txt ALLOWs use of requirements specifiers: https://pip.pypa.io/en/stable/reference/requirements-file-format/

  https://pip.pypa.io/en/stable/reference/requirement-specifiers/#requirement-specifiers

  "Generally speaking, a requirement specifier is composed of a project name followed by optional version specifiers."
  
- exact version format required in a conda spec?

** [2024-09-12] next steps

- [X] update README.
- [X] update CLI docs
- [X] update dry run mode
- [ ] verify that TOML syntax for python version and pip versions matches conda install spec and reqs.txt syntax
- [X] enhance fork into environment with components installed through pip and conda

** [2024-09-20 Fri 20:33]

run_ffmpeg:
- run as executable, triggering pythonrunscript.py:
  - BROKEN
  - incorrectly tries to interpret the file not as python?
  - does not interpreter just as if called by the python3 in the env
    - but: calling =conda run -p /Users/alexis/Library/Caches/pythonrunscript/60b45f611c8f43cc4df61e1bd5157fd8/condaenv python3 run_ffmpeg.py= works

- run from python3:
  - works: runs as desired, as if =ffmpeg -version= was called directly
- run from pythonrunscript.py:
  - works: runs as desired, as if =ffmpeg -version= was called directly

run_conda.py:
- run from python3:
  - 
- run as executable:

  
** [2024-09-23 Mon 22:29] status

- what behavior is desired?
  - 
- does workaround improve behavior?

NO WRAPPER:

test of =ffmpeg=:
- =ffmpeg=:
  - exits, with code 1
- =./run_ffmpeg=:
  -  exits with code 1, and a message emitted by conda saying:
  - BAD: discrepancy. conda inserts a message
- =pythonrunscript.py run_ffmpeg.py=:
  - exits with code 1
  - BAD: and a message emitted by conda saying. (same)

test of =ffmpeg -version=:
- =ffmpeg -version=:
  - exits, with code 0
- =./run_ffmpeg -version=:
  - exits, with code 0
  - BAD: conda intercepts the command line argument.
- =pythonrunscript.py run_ffmpeg.py -version=:
  - BAD: conda intercepts the command line argument.

test of =python3=:
- =./python3=:
  - opens interactive prompt
- =./run_python3.py=:
  - VERY VERY BAD: exits immediately
- =pythonrunscript ./run_python3.py=:
  - without wrapper: opens interactive prompt
  - with wrapper: 
- =python3 ./run_python3.py=:
  - opens interactive prompt
  
test of =python3 -V=:
- =./python3 -V=:
  - works
- =./run_python3.py -V=:
  - works
- =pythonrunscript ./run_python3.py -V=:
  - works

WITH WRAPPER:

- Finding: wrapper SUCCEEDS in fixing the problem that with -version.
- wrapper does not prevent interpreter for quitting rather than opening a terminal
- but wrapper did not cause that problem



summary of non-transparencies:
- =conda run= inserts a message when exit code == 1
- conda intercepts command line arguments like "-version"
  - fixed with wrapper
- conda fails to run an interactive prommpt
  - fixed with command line arg to conda --no-capture-output


#+begin_src
ERROR conda.cli.main_run:execute(125): `conda run /Users/alexis/Library/Caches/pythonrunscript/60b45f611c8f43cc4df61e1bd5157fd8/condaenv/bin/python3 ./run_ffmpeg.py` failed. (See above for error)
#+end_src

** summary of results of work to be perfectly transparent to conda

- scripts can now call conda-installed executable
- worked around a known conda bug

- one non-transparency, is that if pythonrunscript is used to run a
  script in a conda environment, which exits with a nonzero exit code,
  then conda will print some additional logging at exit time.


** [2024-10-07 Mon 15:43]

next steps:

- [X] re-run examples as tests
- [ ] add meatier examples?
- [X] update versioning metadata
- [x] add versioning to CLT  
- [X] copy script to binary
- [ ] deploy to pypi

** [2024-10-07 Mon 19:02] plan to use twine to upload

poetry build
poetry publish