Skip to content

Add basic S3 object retrieval to the pymatgen user agent #4438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

esoteric-ephemera
Copy link
Contributor

The MP API has changed to accommodate increasingly large datasets. This PR allows for retrieving certain data objects from S3 that are no longer accessible via MongoDB

In the future, communicating which features unexpectedly no longer work, e.g., through matsci.org, is a great way to handle feature changes. Calling out the team is not going to help

fyi @mkhorton, @computron

@shyuep
Copy link
Member

shyuep commented Jun 17, 2025

  1. I already communicated the problem to the MP team via email. Got a non helpful response.
  2. Breaking an API with zero communication to the community (and no updates to available documentation) is a bad practice and dangerously careless to begin with.
  3. I don't consider the commit message as "calling out the MP team". Merely stating a fact.

@esoteric-ephemera
Copy link
Contributor Author

esoteric-ephemera commented Jun 17, 2025

  1. Can't comment other than our python client is kept up to date with the most recent database / S3 changes - if a feature set breaks, looking there is a good idea

  2. Let's not be hyperbolic, there's no danger here. Yes, we'd love to be able to update documentation more quickly, but it's much less of a priority given time constraints than working on new data products.

  3. Toeing the line of the code of conduct. The fact is that the same data is available, just no longer in the database for reasons of scale

@shyuep
Copy link
Member

shyuep commented Jun 17, 2025

The API and the python client are two separate entities. Breaking the API is even worse than breaking the python client since an end user is technically supposed to pin to a specific python dependency. An API breakage breaks even previous python clients, i.e., a backwards incompatibility that cannot be mitigated even by pinning. There is a whole literature on API versioning best practices, which was not done in this case.

Documentation and features go hand in hand. Documentation should be added/updated as new data/features are added, not patched as an afterthought when someone finds breakages in downstream code. It is 2025. If the belief is that new features are more important than documentation even in this day, we will have to agree to disagree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants