Skip to content

/ Python

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added stackoverflow python job crawler, Get all of job links that con… #3839

Open
wants to merge 1 commit into
base: master
from
Open

Added stackoverflow python job crawler, Get all of job links that con… #3839

wants to merge 1 commit into from

Conversation

@sonechca
Copy link

@sonechca sonechca commented Oct 30, 2020

…sist of JSON format

Describe your change:

  • [x ] Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms have a URL in its comments that points to Wikipedia or other similar explanation.
  • If this pull request resolves one or more open issues then the commit message contains Fixes: #{$ISSUE_NO}.
@sonechca
Added stackoverflow python job crawler, Get all of job links that con…
Loading status checks…
2b3ef24 
…sist of JSON format
@sonechca sonechca requested a review from cclauss as a code owner Oct 30, 2020
@mrmaxguns
Copy link
Contributor

@mrmaxguns mrmaxguns commented Nov 4, 2020

Here is why the pre-commit failed:

  1. Fix end of files: Your file must end with one empty line
  2. Trim trailing whitespace: Make sure you have no tabs or spaces at the end of lines. Blank lines should have no tabs or spaces.
  3. Black: make sure to run the black linter. Black will automatically fix most of the issues described in this list.
  4. Flake8: there were two errors:
    1. web_programming/crawl_stackoverflow_job.py:5:7: F541 f-string is missing placeholders: your f-string on line 5 is missing placeholders. Replace it with a regular string.
    2. web_programming/crawl_stackoverflow_job.py:8:1: E302 expected 2 blank lines, found 1: you need two blank lines before the function.
@mrmaxguns
mrmaxguns reviewed Nov 4, 2020
View changes
web_programming/crawl_stackoverflow_job.py
from bs4 import BeautifulSoup #extract Stackoverflow pages

#Get the Stackoverflow URL
URL = f"https://stackoverflow.com/jobs?q=python&sort=i"

This comment has been minimized.

Sign in to view
@mrmaxguns

mrmaxguns Nov 4, 2020
Contributor

Change the f-string to a regular string
web_programming/crawl_stackoverflow_job.py

if __name__ == "__main__":
print("Python job finding now...")
get_jobs()

This comment has been minimized.

Sign in to view
@mrmaxguns

mrmaxguns Nov 4, 2020
Contributor

Add a blank line at the end of the file

This comment has been minimized.

Sign in to view
@cclauss

cclauss Nov 4, 2020
Member

Will psf/black add this automatically?

This comment has been minimized.

Sign in to view
@mrmaxguns

mrmaxguns Nov 4, 2020
Contributor

I believe so
web_programming/crawl_stackoverflow_job.py

#Get the Stackoverflow URL
URL = f"https://stackoverflow.com/jobs?q=python&sort=i"

This comment has been minimized.

Sign in to view
@mrmaxguns

mrmaxguns Nov 4, 2020
Contributor

Add another empty line before get_last_page

This comment has been minimized.

Sign in to view
@cclauss

cclauss Nov 4, 2020
Member

Will psf/black add this automatically?

This comment has been minimized.

Sign in to view
@mrmaxguns

mrmaxguns Nov 4, 2020
Contributor

I believe it will
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrmaxguns mrmaxguns

@cclauss cclauss

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
@sonechca @mrmaxguns @cclauss
You can’t perform that action at this time.