Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: nltk/nltk
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: develop
Choose a base ref
...
head repository: ExplodingCabbage/nltk
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: develop
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 17 commits
  • 2 files changed
  • 1 contributor

Commits on Jan 9, 2016

  1. Configuration menu
    Copy the full SHA
    e89631a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    3af726f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9d72aa4 View commit details
    Browse the repository at this point in the history

Commits on Jan 10, 2016

  1. [PorterStemmer] Remove stem_word from PorterStemmer. Breaks backwards…

    … compatability!
    
    Prior to this change, the public API of the PorterStemmer was a mess. NLTK's version was
    based off Vivake Gupta's implementation at http://tartarus.org/~martin/PorterStemmer/python.txt,
    endorsed by Martin himself at http://tartarus.org/~martin/PorterStemmer/. However,
    Gupta's implementation is a shoddy port of Martin Porter's own implementation in C,
    and had several vestigial quirks lying around. These include the claim that the stem()
    method takes a "char pointer" as an argument (no such thing in Python) and the need
    to pass in start and end indexes between which stem() should read the word from the
    given char array.
    
    At some point in nltk's history, during or prior to the 2006 commit that added porter.py
    to the current Git repository:
    
        edf4677
    
    this was "solved" by renaming Vivake's stem() method to stem_word() and creating
    a wrapper for it called stem() that conformed to the StemmerI interface.
    
    This was completely pointless; the right thing to do would've been to remove the
    unnecessary parts of Vivake's stem() method and thereby acheive conformity to StemmerI.
    This commit does this, but at the cost of breaking backwards compatibility for
    anyone who was using stem_word(word) instead of stem(word); those people will
    need to adjust their application code when updating to the latest version of NLTK.
    ExplodingCabbage committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    2000554 View commit details
    Browse the repository at this point in the history
  2. [PorterStemmer] Remove more commented out code.

    I don't even know where this came from. It's not part of the Vivake Gupta version
    at http://tartarus.org/~martin/PorterStemmer/python.txt
    
    Regardless, there's no reason for it to remain here.
    ExplodingCabbage committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    8b7ffe6 View commit details
    Browse the repository at this point in the history
  3. [PorterStemmer] Remove _adjust_case for consistency with Lancaster an…

    …d Snowball
    
    Handling of upper and lower case is not specified in Martin Porter's
    "An algorithm for suffix stripping" paper; the algorithm description
    there never even mentions the existence of difference letter cases.
    Nor does Martin's C implementation of the stemmer at:
    
       http://tartarus.org/~martin/PorterStemmer/c.txt
    
    handle case in the way that NLTK's version has been doing; instead,
    it simply requires that the user convert their word to lowercase
    before calling stem().
    
    Since there is no Porter-specific reason to preserve our (odd)
    behaviour here, and our other StemmerI implementations don't do it,
    we should probably purge it, as this commit does.
    ExplodingCabbage committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    a2d1dfa View commit details
    Browse the repository at this point in the history
  4. [PorterStemmer] Reimplement steps 1a and 1b while:

    - Making the code more readable and including quotes in comments that allow the code
      to be matched up with Porter's published algorithm
    - Marking NLTK-specific departure points properly
    ExplodingCabbage committed Jan 10, 2016
    Configuration menu
    Copy the full SHA
    829b29a View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    221c193 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    ccd5525 View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    10498f8 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    92d845f View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    9ad2b28 View commit details
    Browse the repository at this point in the history
  10. Configuration menu
    Copy the full SHA
    cb160de View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    fab16ba View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    dccf396 View commit details
    Browse the repository at this point in the history

Commits on Jan 11, 2016

  1. [PorterStemmer] Create 'modes' to let users use a faithful version of…

    … the algorithm. Also sort out the docs.
    ExplodingCabbage committed Jan 11, 2016
    Configuration menu
    Copy the full SHA
    e7ec0ba View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0cc39c6 View commit details
    Browse the repository at this point in the history
Loading