Removed API dependency for Tags by RohanJnr · Pull Request #803 · python-discord/bot

RohanJnr · 2020-02-29T09:55:58Z

closes #777

This PR is to remove the API dependency for the Tags and use static files instead, so adding, editing or deleting of a tag will happen only through a PR and not from a discord bot command.

Added all the static tag files in the resources folder.
Added white spaces on the lines before the bullet points for proper GitHub rendering of .md files.
Modified cogs/tags.py to use the static files rather than using the API. Removed all methods which call the API for adding, editing or deleting tags.

…o access the static tag files rather than sending an API get request. Removed all methods calling the API so the tags cannot be edited, added nor deleted.

…ering of points on github

…d of caching it after the tags command is used.

…tags_overhaul

lemonsaurus

Please rewrite this to make full use of pathlib instead of leaning on the os module to solve most of the problems for you. we shouldn't need to import os at all for this file.

The pathlib module simplifies opening and reading files, hence the os module and the context manager are no longer used.

"get_tags()" and "_get_tag()" functions need not be async as we are no longer doing any API call but instead reading from local files.

MarkKoz · 2020-03-04T22:02:11Z

Hey @RohanJnr, recently more pre-commit hooks were added. I'm not sure if you don't have the pre-commit hook installed or if it just failed to update and include the new hooks for you. You may need to merge master first. CI is failing because the new hooks remove trailing whitespaces and ensure final newlines in files. Some of the tag markdown files you added have those issues. If you run pipenv run lint and commit the changes, it should fix all those issues.

sco1 · 2020-03-04T22:35:07Z

The new hooks have been merged & the tag files updated for the new linting. Please be sure to pull down the changes before making more commits, and ensure that your linting environment has been updated appropriately.

lemonsaurus

Looks good.

MarkKoz

Since there's only one command now, a command group is no longer necessary. Be mindful of existing aliases in alias.py when you go to remove the group.

…tags_overhaul

sco1 · 2020-03-12T17:51:43Z

+
+For reference, this usage is covered by the following clauses in [YouTube's TOS](https://www.youtube.com/static?template=terms), as of 2019-07-22:
+```
+The following restrictions apply to your use of the Service. You are not allowed to:


Not sure why the language regarding redistribution was removed from this tag:

access, reproduce, download, distribute, transmit, broadcast, display, sell, license, alter, modify or otherwise use any part of the Service or any Content except: (a) as expressly authorized by the Service; or (b) with prior written permission from YouTube and, if applicable, the respective rights holders;

use the Service to view or listen to Content other than for personal, non-commercial use (for example, you may not publicly screen videos or stream music from the Service); or

I removed it because it is just a side effect of youtube-dl, this tool in itself is only against 3rd paragraph, the first and 9th paragraph are about redistributibg the content created by tool such as youtube-dl, but aren't directly caused by it. For instance I could download my music collection using it and keep it on my computer, and it would only be against the 3rd paragraph.

The overwhelming majority of use cases encountered on our server utilize ytdl to stream content with a discord bot, which is redistributing content outside of YouTube’s platform. That is why these sections were all included and why they should remain in the tag.

Please ask questions on these things before unilaterally making a decision to remove them.

It just didn't made sense to me based on what was already in the tag.

I've added 1 & 9.

MarkKoz

The tag search command is broken:

AttributeError: 'Tags' object has no attribute '_get_tags'

As far as I can see, everything else is fine. Fix the above error and this PR should be good to go.

The tags have now been shifted from the database to being static files and hence the get_tags() method has undergone changes. It now dosen't fetch from the database but looks at the local files and we need not call it more than once.

* Add a constant for the footer text * Import constants module rather than its classes

…tags_overhaul

Fixed `!tag search any` raises `AttributeError`. Changed default value of `keywords` from `None` to `'any'`. This will make it search for keyword `'any'` when there is no keyword.

RohanJnr added 3 commits February 29, 2020 14:29

Added all the tag files in resources and modified cogs/tags.py file t…

9fd7b08

…o access the static tag files rather than sending an API get request. Removed all methods calling the API so the tags cannot be edited, added nor deleted.

added white spaces on statements before bullet points for proper rend…

f256346

…ering of points on github

Re-corrected the lines which I had changed by mistake

fe31808

RohanJnr requested a review from a team as a code owner February 29, 2020 09:55

RohanJnr requested review from MarkKoz and tagptroll1 and removed request for a team February 29, 2020 09:55

Numerlor suggested changes Feb 29, 2020

View reviewed changes

Comment thread bot/cogs/tags.py

Merge branch 'master' into tags_overhaul

d583e9b

sco1 mentioned this pull request Feb 29, 2020

Wrong annotation for Tags._fuzzy_search #802

Closed

RohanJnr added 2 commits February 29, 2020 21:48

Caching all the tags when the bot has loaded(caching only once) inste…

1b56868

…d of caching it after the tags command is used.

Merge branch 'tags_overhaul' of https://github.com/RohanJnr/bot into …

a7ad0b7

…tags_overhaul

ikuyarihS suggested changes Mar 1, 2020

View reviewed changes

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

MarkKoz added a: information Related to information commands: (doc, help, information, reddit, site, tags) p: 2 - normal Normal Priority s: waiting for author Waiting for author to address a review or respond to a comment t: feature New feature or request labels Mar 1, 2020

This was referenced Mar 1, 2020

Fix grammar and formatting in tags. python-discord/meta#39

Closed

Create the "Generators" tag python-discord/meta#35

Closed

lemonsaurus changed the title ~~Removed API reliability for Tags~~ Removed API dependency for Tags Mar 3, 2020

lemonsaurus requested changes Mar 3, 2020

View reviewed changes

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

RohanJnr added 2 commits March 4, 2020 23:30

Use "pathlib" instead of "os" module and context manager

1c7b2d8

The pathlib module simplifies opening and reading files, hence the os module and the context manager are no longer used.

Convert "get_tags()" and "_get_tag()" to sync functions

073fdf1

"get_tags()" and "_get_tag()" functions need not be async as we are no longer doing any API call but instead reading from local files.

sco1 added 2 commits March 4, 2020 17:19

Merge branch 'master' into tags_overhaul

c624c25

Update tag files for new linting hooks

564690f

RohanJnr requested review from ikuyarihS and lemonsaurus March 8, 2020 14:36

sco1 mentioned this pull request Mar 8, 2020

Write unit tests for bot/cogs/tags.py #589

Closed

lemonsaurus approved these changes Mar 9, 2020

View reviewed changes

MarkKoz requested changes Mar 9, 2020

View reviewed changes

Comment thread bot/cogs/tags.py Outdated

Comment thread bot/cogs/tags.py Outdated

Update ytdl tag to the new YouTube ToS

4662cfd

RohanJnr force-pushed the tags_overhaul branch from df15933 to 4662cfd Compare March 12, 2020 16:40

RohanJnr added 2 commits March 12, 2020 22:18

remove repetitive file search

f2d10e4

Merge branch 'master' into tags_overhaul

9bc6744

RohanJnr requested a review from MarkKoz March 12, 2020 16:56

RohanJnr added 2 commits March 12, 2020 22:32

convert get_tags() method to staticmethod

55effb3

Merge branch 'tags_overhaul' of https://github.com/RohanJnr/bot into …

d56639c

…tags_overhaul

sco1 reviewed Mar 12, 2020

View reviewed changes

MarkKoz requested changes Mar 13, 2020

View reviewed changes

RohanJnr and others added 3 commits March 14, 2020 19:43

Remove line that calls get_tags() method

e9ae8a8

The tags have now been shifted from the database to being static files and hence the get_tags() method has undergone changes. It now dosen't fetch from the database but looks at the local files and we need not call it more than once.

Tags: use constant for command prefix in embed footer

52ed9aa

* Add a constant for the footer text * Import constants module rather than its classes

Tags: add restrictions 1 & 9 from YouTube ToS to ytdl tag

eeaccea

MarkKoz approved these changes Mar 14, 2020

View reviewed changes

ikuyarihS suggested changes Mar 15, 2020

View reviewed changes

Comment thread bot/cogs/tags.py

RohanJnr and others added 4 commits March 15, 2020 17:05

convert _get_tags_via_content() method to non-async

5865126

Merge branch 'tags_overhaul' of https://github.com/RohanJnr/bot into …

156492c

…tags_overhaul

not awaiting _get_tags_via_content() method as it is non-async

520c730

Fixed tag search via contents, any keywords.

8eea9c3

Fixed `!tag search any` raises `AttributeError`. Changed default value of `keywords` from `None` to `'any'`. This will make it search for keyword `'any'` when there is no keyword.

ikuyarihS approved these changes Mar 15, 2020

View reviewed changes

ikuyarihS merged commit 99c05dc into python-discord:master Mar 15, 2020

Numerlor mentioned this pull request Mar 21, 2020

Wrong annotation for Tags._fuzzy_search #789

Closed

Xithrius removed the s: waiting for author Waiting for author to address a review or respond to a comment label May 27, 2022

Uh oh!

Uh oh!

Conversation

RohanJnr commented Feb 29, 2020 • edited by lemonsaurus Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lemonsaurus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MarkKoz commented Mar 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sco1 commented Mar 4, 2020

Uh oh!

lemonsaurus left a comment

Choose a reason for hiding this comment

Uh oh!

MarkKoz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sco1 Mar 12, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Akarys42 Mar 12, 2020

Choose a reason for hiding this comment

Uh oh!

sco1 Mar 12, 2020

Choose a reason for hiding this comment

Uh oh!

Akarys42 Mar 12, 2020

Choose a reason for hiding this comment

Uh oh!

MarkKoz Mar 14, 2020

Choose a reason for hiding this comment

Uh oh!

MarkKoz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

RohanJnr commented Feb 29, 2020 •

edited by lemonsaurus

Loading

MarkKoz commented Mar 4, 2020 •

edited

Loading

sco1 Mar 12, 2020 •

edited

Loading