Skip to content

Create distributed quick sort algorithm#3100

Closed
zhe-thoughts wants to merge 2 commits intoTheAlgorithms:masterfrom
zhe-thoughts:patch-1
Closed

Create distributed quick sort algorithm#3100
zhe-thoughts wants to merge 2 commits intoTheAlgorithms:masterfrom
zhe-thoughts:patch-1

Conversation

@zhe-thoughts
Copy link

@zhe-thoughts zhe-thoughts commented Oct 9, 2020

This algorithm extends from quick_sort.py in this same directory. It uses Ray to make quick sort execute in distributed manner.

On my local machine, it has 3x speedup:

Sequential execution: 16.010289192199707
Distributed execution: 5.844448089599609

Describe your change:

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms have a URL in its comments that points to Wikipedia or other similar explanation.
  • If this pull request resolves one or more open issues then the commit message contains Fixes: #{$ISSUE_NO}.

This algorithm extends from quick_sort.py in this same directory. It uses Ray to make quick sort execute in distributed manner.

On my local machine, it has 3x speedup:

Sequential execution: 16.010289192199707
Distributed execution: 5.844448089599609
@zhe-thoughts
Copy link
Author

NOTE: this algorithm extends from the existing quick_sort.py in the same directory. Please let me know how we usually handles this (should I import it? seems an overkill since the code is very simple)

@TravisBuddy
Copy link

Travis tests have failed

Hey @zhe-thoughts,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: 54c59f00-09ff-11eb-b8c5-47e0b209ea74

@seelam
Copy link

seelam commented Oct 18, 2020

I gave this a try... all looks good but the WARNINGS are annoying and it will be good to limit the concurrency via an API or config call:

ubuntu@seelam-ansible-host:~$ python quicksort.py
....
Sequential execution: 20.511595964431763
2020-10-18 22:02:01,225 WARNING worker.py:1047 -- WARNING: 24 PYTHON workers have been started. This could be a result of using a large number of actors, or it could be a consequence of using nested tasks (see ray-project/ray#3644) for some a discussion of workarounds.
....
Distributed execution: 10.588677167892456

In your case, since you are using the regular quick sort for <200K elements and total elements are 1M, only ~10 actors. When I changed the 200k to 20k, the distributed version took 4x longer than sequential version:
Distributed execution: 84.4353358745575

I guess it is important to figure out at what granularity of computation, you need to use the distributed version.

@stale
Copy link

stale bot commented Nov 21, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale Used to mark an issue or pull request stale. label Nov 21, 2020
@stale
Copy link

stale bot commented Dec 10, 2020

Please reopen this pull request once you commit the changes requested or make improvements on the code. If this is not the case and you need some help, feel free to seek help from our Gitter or ping one of the reviewers. Thank you for your contributions!

@stale stale bot closed this Dec 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Used to mark an issue or pull request stale.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants