From 96dbb9edd1eb7de13e6eff85f1353c0804335a7d Mon Sep 17 00:00:00 2001 From: Mohseen Mukaddam Date: Wed, 9 Aug 2017 16:01:56 -0700 Subject: [PATCH 1/4] adding manual readme --- generate/templates/README.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 generate/templates/README.md diff --git a/generate/templates/README.md b/generate/templates/README.md new file mode 100644 index 000000000..b1f7702f0 --- /dev/null +++ b/generate/templates/README.md @@ -0,0 +1,34 @@ +# Manual templates + + Manual templates override generated code from nodegit while generating source code. + +## Why? + +If generated code does not accurately wrap the libgit2 calls, you might want to consider implementing manual templates in the following cases: + + #### 1. Convert Sync functions to Async + > typically this can be done by adding `"isAsync": true` in **descriptor.json**, but if you wish to not expose all parameters, customize/wrap new arguments, manual templates are the faster way to achieve it. + + #### 2. Segfaults non-deterministically + > If that is the case, use the persistence pattern as seen in x, y wrappers. Garbage collector is probably eating up data that was not persisted after the function scope ends. + +
+----- +## Implementing manual templates + +#### 1. Copy generated .cc and .h files to *generate/templates/manual/* + *.cc files -> generate/templates/manual/src/ + *.h files -> generate/templates/manual/include/ + +#### 2. Remove all references from json configuration files (descriptor, libgit2-supplement) + +#### 3. Add references to binding.gyp template + location: `generate/templates/templates/binding.gyp` + +#### 4. Add headers to nodegit.cc template + location: `generate/templates/templates/nodegit.cc` + +#### 5. Add new wrapper to nodegit.js template + use rawApi.ManualWrapper reference to add _ManualWrapper + add any js wrapper (if any) via importExtension + From f7cfe3277d91be646324b36199734813219d9428 Mon Sep 17 00:00:00 2001 From: Mohseen Mukaddam Date: Wed, 9 Aug 2017 17:05:23 -0700 Subject: [PATCH 2/4] Readmes for examples, lib, generate, lifecycleScripts --- examples/README.md | 4 ++++ generate/README.md | 21 +++++++++++++++++++++ lib/README.md | 3 +++ lifecycleScripts/README.md | 5 +++++ 4 files changed, 33 insertions(+) create mode 100644 examples/README.md create mode 100644 generate/README.md create mode 100644 lib/README.md create mode 100644 lifecycleScripts/README.md diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 000000000..1b1fc2b32 --- /dev/null +++ b/examples/README.md @@ -0,0 +1,4 @@ +## /examples + + List of examples of using NodeGit to implement common git core operations. + diff --git a/generate/README.md b/generate/README.md new file mode 100644 index 000000000..504824fd5 --- /dev/null +++ b/generate/README.md @@ -0,0 +1,21 @@ +## Generate + +The scripts and templates in this dir, help generate the source code and tests for NodeGit. The major components of generate are: + + 1. Input + 2. Scripts + 3. Templates + +### Input + + All the **configuration** required for the parser to generate the source code and tests. For more details, check the Input directory. + +### Scripts + + The scripts that generate the final configuration (*snapshot of the library*) `idefs.json`, `missing-tests.json`. These configurations are then used to generate `src` for the library. + +### Templates + + All the Combyne templates are placed here. The filters, partials, templates all help NodeGit generate the source code. + +// TODO: link to combyne, tutorial and docs. diff --git a/lib/README.md b/lib/README.md new file mode 100644 index 000000000..d60bcc450 --- /dev/null +++ b/lib/README.md @@ -0,0 +1,3 @@ +## /lib + + Contains wrappers to abstract the internals of the JS part of NodeGit Module. diff --git a/lifecycleScripts/README.md b/lifecycleScripts/README.md new file mode 100644 index 000000000..4a9fb06be --- /dev/null +++ b/lifecycleScripts/README.md @@ -0,0 +1,5 @@ +## /lifecycleScripts + + These scripts are responsible for downloading the right dependencies, configuring vendors, and all other dependencies that are required to build, generate and clean the module. + + \ No newline at end of file From 71685d76d3b85259dc171091a42763f5c778ac69 Mon Sep 17 00:00:00 2001 From: Mohseen Mukaddam Date: Thu, 10 Aug 2017 13:09:59 -0700 Subject: [PATCH 3/4] Adding docs for NodeGit --- generate/README.md | 4 ++-- generate/input/README.md | 15 +++++++++++++++ generate/templates/README.md | 23 +++++++++++++---------- lib/README.md | 2 +- test/README.md | 17 +++++++++++++++++ utils/README.md | 6 ++++++ 6 files changed, 54 insertions(+), 13 deletions(-) create mode 100644 generate/input/README.md create mode 100644 test/README.md create mode 100644 utils/README.md diff --git a/generate/README.md b/generate/README.md index 504824fd5..e39814e16 100644 --- a/generate/README.md +++ b/generate/README.md @@ -1,4 +1,4 @@ -## Generate +## /generate The scripts and templates in this dir, help generate the source code and tests for NodeGit. The major components of generate are: @@ -18,4 +18,4 @@ The scripts and templates in this dir, help generate the source code and tests f All the Combyne templates are placed here. The filters, partials, templates all help NodeGit generate the source code. -// TODO: link to combyne, tutorial and docs. + > For more information on Combyne: [tbranyen/combyne](https://github.com/tbranyen/combyne) diff --git a/generate/input/README.md b/generate/input/README.md new file mode 100644 index 000000000..09f2afb17 --- /dev/null +++ b/generate/input/README.md @@ -0,0 +1,15 @@ +## /generate/input + This folder contains the main config files to generate NodeGit + + #### Callbacks + Add all meta data about the callbacks from libgit2 that need to be implemented in NodeGit + + #### Descriptor + Customize the generated code using this configuration. Enter the function signature, arguments and their metadata and which functions can be skipped in this file. If you're using a manual template, remove all its references from this file. + + #### Libgit2-docs + These are provided by the libgit2 team. Includes all metadata about the api provided by the libgit2 library. + + #### Libgit2-supplement + Use this confiuration file to group and override parts of the generated code. NodeGit tries it's best to generate the right classes and structs, if that is not the case, use this config file to group/remove the functions. + > If you're using manual templates, update the `cFile` reference to point to the manual template diff --git a/generate/templates/README.md b/generate/templates/README.md index b1f7702f0..b14508ff7 100644 --- a/generate/templates/README.md +++ b/generate/templates/README.md @@ -1,32 +1,35 @@ # Manual templates - Manual templates override generated code from nodegit while generating source code. + Manual templates override generated code from nodegit while generating source code. They really should be avoid untill absolutely necessary. ## Why? If generated code does not accurately wrap the libgit2 calls, you might want to consider implementing manual templates in the following cases: - #### 1. Convert Sync functions to Async - > typically this can be done by adding `"isAsync": true` in **descriptor.json**, but if you wish to not expose all parameters, customize/wrap new arguments, manual templates are the faster way to achieve it. + #### 1. Performance + > Everytime the library switches between the C land and the JS queue thread, there is a penalty in performance. If the generated code switches frequently, it might be better option to use manual templates. - #### 2. Segfaults non-deterministically - > If that is the case, use the persistence pattern as seen in x, y wrappers. Garbage collector is probably eating up data that was not persisted after the function scope ends. + #### 2. Saftey + > The generated code sometimes does not handle structures that are inter-dependant. Perfect example would be convenient_hunks. Hunks references a file pointer and diff lines that are dependant on it. If persisted, it would lock the file. If garbage collected, the diff lines would cause seg fault errors. Anytime a custom solution is required, that would be hard for generated code to implement, manual templates should be used. + + #### 3. Odd cases + > If a new pattern exists in libgit that would be difficult to implement using generated code, manual templates can be used for one-off cases. Typically generated code takes care of most patterns seen in libgit, but if function signatures do not follow typical pattern, manual templates could be used. Example: git_filter.
----- ## Implementing manual templates #### 1. Copy generated .cc and .h files to *generate/templates/manual/* - *.cc files -> generate/templates/manual/src/ - *.h files -> generate/templates/manual/include/ + *.cc files -> /generate/templates/manual/src/ + *.h files -> /generate/templates/manual/include/ -#### 2. Remove all references from json configuration files (descriptor, libgit2-supplement) +#### 2. Remove all references from /generate configuration files #### 3. Add references to binding.gyp template - location: `generate/templates/templates/binding.gyp` + location: /generate/templates/templates/binding.gyp #### 4. Add headers to nodegit.cc template - location: `generate/templates/templates/nodegit.cc` + location: /generate/templates/templates/nodegit.cc #### 5. Add new wrapper to nodegit.js template use rawApi.ManualWrapper reference to add _ManualWrapper diff --git a/lib/README.md b/lib/README.md index d60bcc450..fab416598 100644 --- a/lib/README.md +++ b/lib/README.md @@ -1,3 +1,3 @@ ## /lib - Contains wrappers to abstract the internals of the JS part of NodeGit Module. + Contains wrappers to abstract the internals of the JavaScript part of NodeGit Module. diff --git a/test/README.md b/test/README.md new file mode 100644 index 000000000..3964c0410 --- /dev/null +++ b/test/README.md @@ -0,0 +1,17 @@ +## /test + + Contains all the test scripts, runner and keys for running the tests. + + ----------- + + #### /home + contains gitconfig for the test repositories. + + #### /repos + contains blame, empty, nonrepo and workdir test repositories. + + #### /tests + unit tests for NodeGit. + + #### /utils + test utilities with garbage collector, index and repository setup, that can be used in tests. diff --git a/utils/README.md b/utils/README.md new file mode 100644 index 000000000..4b74d5111 --- /dev/null +++ b/utils/README.md @@ -0,0 +1,6 @@ +## /utils + + contains utilities for NodeGit + + #### buildFlags + determines how NodeGit should build. Use `BUILD_ONLY` environment variable to build from source. From 0e2dffadabe6ee88f4b7f7794224100b74bc0189 Mon Sep 17 00:00:00 2001 From: mohseenrm Date: Thu, 17 Aug 2017 12:47:46 -0700 Subject: [PATCH 4/4] Adding review changes --- generate/README.md | 2 +- generate/templates/README.md | 10 ++++------ lib/README.md | 2 +- lifecycleScripts/README.md | 2 +- test/README.md | 10 +++++----- utils/README.md | 4 ++-- 6 files changed, 14 insertions(+), 16 deletions(-) diff --git a/generate/README.md b/generate/README.md index e39814e16..3ee5446d6 100644 --- a/generate/README.md +++ b/generate/README.md @@ -1,6 +1,6 @@ ## /generate -The scripts and templates in this dir, help generate the source code and tests for NodeGit. The major components of generate are: +The scripts and templates in this dir help generate the source code and tests for NodeGit. The major components of generate are: 1. Input 2. Scripts diff --git a/generate/templates/README.md b/generate/templates/README.md index b14508ff7..ee8b1f70c 100644 --- a/generate/templates/README.md +++ b/generate/templates/README.md @@ -4,22 +4,20 @@ ## Why? -If generated code does not accurately wrap the libgit2 calls, you might want to consider implementing manual templates in the following cases: - #### 1. Performance - > Everytime the library switches between the C land and the JS queue thread, there is a penalty in performance. If the generated code switches frequently, it might be better option to use manual templates. + > Everytime the library switches between C land and the javascript thread, there is a penalty in performance. If in practice the usage of a method in libgit2 requires crossing the c/javascript boundary frequently, it might be better option to use manual templates. An example being ```Revwalk::FastWalk```. #### 2. Saftey - > The generated code sometimes does not handle structures that are inter-dependant. Perfect example would be convenient_hunks. Hunks references a file pointer and diff lines that are dependant on it. If persisted, it would lock the file. If garbage collected, the diff lines would cause seg fault errors. Anytime a custom solution is required, that would be hard for generated code to implement, manual templates should be used. + > The generated code sometimes does not handle structures that are interdependent. An example would be ```git_patch``` and ```git_diff```. A ```git_patch```'s memory is owned by ```git_diff```, and that includes all of the children of ```git_patch```, as well. So a ```git_diff_hunk```, ```git_diff_line```, and ```git_patch``` all are owned by a ```git_diff```, and when that ```git_diff``` is deleted, all the memory for any patches, hunks, or lines that are in use as NodeGitWrappers are now corrupted. Further, a ```git_diff``` keeps a file handle open for its entire lifespan, which can lead to NodeGit holding onto file locks in Windows. Due to both of these compounding issues, we wrote manual templates to shift ownership away from a ```git_diff``` to ```git_patch```, ```git_diff_hunk```, and ```git_diff_line``` and also shorten the lifespan of a diff. #### 3. Odd cases - > If a new pattern exists in libgit that would be difficult to implement using generated code, manual templates can be used for one-off cases. Typically generated code takes care of most patterns seen in libgit, but if function signatures do not follow typical pattern, manual templates could be used. Example: git_filter. + > If a new pattern exists in libgit that would be difficult to implement using generated code, manual templates can be used for one-off cases. Typically generated code takes care of most patterns seen in libgit, but if function signatures do not follow typical pattern, manual templates could be used. Example: ```git_filter``` and ```git_remote_ls```.
----- ## Implementing manual templates -#### 1. Copy generated .cc and .h files to *generate/templates/manual/* +#### 1. Write manual .cc and .h files to *generate/templates/manual/* *.cc files -> /generate/templates/manual/src/ *.h files -> /generate/templates/manual/include/ diff --git a/lib/README.md b/lib/README.md index fab416598..3d12686af 100644 --- a/lib/README.md +++ b/lib/README.md @@ -1,3 +1,3 @@ ## /lib - Contains wrappers to abstract the internals of the JavaScript part of NodeGit Module. + Contains javascript extensions for the generated NodeGit modules. Any additional behavior on top of the standard libgit2 behavior will be found here. diff --git a/lifecycleScripts/README.md b/lifecycleScripts/README.md index 4a9fb06be..cc1feeb14 100644 --- a/lifecycleScripts/README.md +++ b/lifecycleScripts/README.md @@ -1,5 +1,5 @@ ## /lifecycleScripts - These scripts are responsible for downloading the right dependencies, configuring vendors, and all other dependencies that are required to build, generate and clean the module. + These scripts are responsible for downloading the right dependencies, configuring vendors, and all other dependencies that are required to build, generate, and clean the module. \ No newline at end of file diff --git a/test/README.md b/test/README.md index 3964c0410..e73556c7f 100644 --- a/test/README.md +++ b/test/README.md @@ -1,17 +1,17 @@ ## /test - Contains all the test scripts, runner and keys for running the tests. + Contains all the test scripts, runner, and keys for running the tests. ----------- #### /home - contains gitconfig for the test repositories. + Contains gitconfig for the test repositories. #### /repos - contains blame, empty, nonrepo and workdir test repositories. + Contains blame, empty, nonrepo, and workdir test repositories. #### /tests - unit tests for NodeGit. + Unit tests for NodeGit. #### /utils - test utilities with garbage collector, index and repository setup, that can be used in tests. + Test utilities with garbage collector, index, and repository setup, that can be used in tests. diff --git a/utils/README.md b/utils/README.md index 4b74d5111..371f3c61f 100644 --- a/utils/README.md +++ b/utils/README.md @@ -1,6 +1,6 @@ ## /utils - contains utilities for NodeGit + Contains utilities for NodeGit #### buildFlags - determines how NodeGit should build. Use `BUILD_ONLY` environment variable to build from source. + Determines how NodeGit should build. Use `BUILD_ONLY` environment variable to build from source.