Skip to content

/ CodeSearchNet

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

question: changing the default leaderboard order #129

Closed
monperrus opened this issue Mar 17, 2020 · 6 comments
Closed

question: changing the default leaderboard order #129

monperrus opened this issue Mar 17, 2020 · 6 comments

Comments

@monperrus
Copy link
Contributor

@monperrus monperrus commented Mar 17, 2020

The default ordering on the leaderboard is "Mean NDCG" over all results that are not None.

This means it is very easy to appear #1 by overfitting on a single language.

What about computing "Mean NDCG" over all 6 languages and replacing None by 0?
@hamelsmu
Copy link
Member

@hamelsmu hamelsmu commented Mar 18, 2020

@staceysv Can you help with this?
@hamelsmu
Copy link
Member

@hamelsmu hamelsmu commented Apr 11, 2020

@staceysv @nbardy can you help with this issue?
@hamelsmu
Copy link
Member

@hamelsmu hamelsmu commented May 3, 2020

Closing this issue as it has become stale
@hamelsmu hamelsmu closed this May 3, 2020
@monperrus
Copy link
Contributor Author

@monperrus monperrus commented May 4, 2020

Ack. I understand that the default leaderboard order will stay the same.
@hamelsmu
Copy link
Member

@hamelsmu hamelsmu commented May 4, 2020

@cvphelps can you look into this questino re: leaderboard ?
@hamelsmu hamelsmu reopened this May 4, 2020
@hamelsmu
Copy link
Member

@hamelsmu hamelsmu commented Sep 4, 2020

This competition has now concluded
@hamelsmu hamelsmu closed this Sep 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@monperrus @hamelsmu
You can’t perform that action at this time.