fix: container env generation for S3 URI and add test for the same by shreyapandit · Pull Request #2971 · aws/sagemaker-python-sdk

shreyapandit · 2022-03-04T18:42:50Z

Description of changes:
Fixes a bug wherein file:// prefix was wrongly appended even for S3 URI Paths

Testing done:
Ran unit tests, integs to follow here

Merge Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.

General

I have read the CONTRIBUTING doc
I certify that the changes I am introducing will be backword compatible, and I have discussed concerns about this, if any, with the Python SDK team
I used the commit message format described in CONTRIBUTING
I have passed the region in to all S3 and STS clients that I've initialized as part of this change.
I have updated any necessary documentation, including READMEs and API docs (if appropriate)

Tests

I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
I have checked that my tests are not configured for a specific region or account (if appropriate)
I have used unique_name_from_base to create resource names in integ tests (if appropriate)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

sagemaker-bot · 2022-03-04T18:54:36Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 48ca435
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T20:00:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 48ca435
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T20:14:47Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 48ca435
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

jeniyat

LGTM

sagemaker-bot · 2022-03-04T23:04:24Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 31528ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T23:18:03Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 845bda5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

codecov-commenter · 2022-03-04T23:23:41Z

Codecov Report

Merging #2971 (9967091) into dev (b9f90dc) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##              dev    #2971   +/-   ##
=======================================
  Coverage   89.81%   89.82%           
=======================================
  Files         196      196           
  Lines       16564    16564           
=======================================
+ Hits        14877    14878    +1     
+ Misses       1687     1686    -1

Impacted Files	Coverage Δ
src/sagemaker/model.py	`91.79% <100.00%> (+0.31%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9f90dc...9967091. Read the comment docs.

sagemaker-bot · 2022-03-04T23:36:30Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 31528ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T23:46:29Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-unit-tests
Commit ID: 9967091
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-04T23:51:35Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 31528ac
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-05T00:22:59Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 845bda5
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-05T00:23:00Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-local-mode-tests
Commit ID: 9967091
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-05T00:34:01Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 845bda5
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-bot · 2022-03-05T00:45:09Z

AWS CodeBuild CI Report

CodeBuild project: sagemaker-python-sdk-slow-tests
Commit ID: 9967091
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

…2971)

* change: update code to get commit_id in codepipeline (#2961) * feature: Data Serializer (#2956) * change: reorganize test files for workflow (#2960) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> * feature: TensorFlow 2.4 for Neo (#2861) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> * fix: Remove sagemaker_job_name from hyperparameters in TrainingStep (#2950) Co-authored-by: Payton Staub <pstaub@amazon.com> * fix: Style update in DataSerializer (#2962) * documentation: smddp doc update (#2968) * fix: container env generation for S3 URI and add test for the same (#2971) * documentation: update sagemaker training compiler docstring (#2969) * feat: Python 3.9 for readthedocs (#2973) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> * fix doc structure * archive 1.6.0 doc * add new args, refs, and links * fix version number * incorp eng feedback, update docstrings, improve xref * Trigger Build * minor fix, trigger build again * fix typo Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: Shreya Pandit <shreya.pandit25@gmail.com> Co-authored-by: Ahsan Khan <ahsan.al.zaki@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>

* change: update code to get commit_id in codepipeline (aws#2961) * feature: Data Serializer (aws#2956) * change: reorganize test files for workflow (aws#2960) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> * feature: TensorFlow 2.4 for Neo (aws#2861) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> * fix: Remove sagemaker_job_name from hyperparameters in TrainingStep (aws#2950) Co-authored-by: Payton Staub <pstaub@amazon.com> * fix: Style update in DataSerializer (aws#2962) * documentation: smddp doc update (aws#2968) * fix: container env generation for S3 URI and add test for the same (aws#2971) * documentation: update sagemaker training compiler docstring (aws#2969) * feat: Python 3.9 for readthedocs (aws#2973) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> * fix doc structure * archive 1.6.0 doc * add new args, refs, and links * fix version number * incorp eng feedback, update docstrings, improve xref * Trigger Build * minor fix, trigger build again * fix typo Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: Shreya Pandit <shreya.pandit25@gmail.com> Co-authored-by: Ahsan Khan <ahsan.al.zaki@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>

…ws#2971)

* change: update code to get commit_id in codepipeline (aws#2961) * feature: Data Serializer (aws#2956) * change: reorganize test files for workflow (aws#2960) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> * feature: TensorFlow 2.4 for Neo (aws#2861) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> * fix: Remove sagemaker_job_name from hyperparameters in TrainingStep (aws#2950) Co-authored-by: Payton Staub <pstaub@amazon.com> * fix: Style update in DataSerializer (aws#2962) * documentation: smddp doc update (aws#2968) * fix: container env generation for S3 URI and add test for the same (aws#2971) * documentation: update sagemaker training compiler docstring (aws#2969) * feat: Python 3.9 for readthedocs (aws#2973) Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> * fix doc structure * archive 1.6.0 doc * add new args, refs, and links * fix version number * incorp eng feedback, update docstrings, improve xref * Trigger Build * minor fix, trigger build again * fix typo Co-authored-by: Navin Soni <navinsoni89@gmail.com> Co-authored-by: Jeniya Tabassum <jeniya.tabassum@gmail.com> Co-authored-by: qidewenwhen <32910701+qidewenwhen@users.noreply.github.com> Co-authored-by: Ben Crabtree <bencrab@amazon.com> Co-authored-by: Dewen Qi <qidewen@amazon.com> Co-authored-by: Qingzi-Lan <83724147+Qingzi-Lan@users.noreply.github.com> Co-authored-by: Payton Staub <staubhpa@gmail.com> Co-authored-by: Payton Staub <pstaub@amazon.com> Co-authored-by: Shreya Pandit <shreya.pandit25@gmail.com> Co-authored-by: Ahsan Khan <ahsan.al.zaki@gmail.com> Co-authored-by: Mufaddal Rohawala <89424143+mufaddal-rohawala@users.noreply.github.com>

Fixes container env generation for S3 URI, update test

48ca435

move test to model dir, update test to test properties based model

31528ac

jeniyat approved these changes Mar 4, 2022

View reviewed changes

shreyapandit changed the title ~~Fixes container env generation for S3 URI, update test~~ Fixes container env generation for S3 URI and add test for the same Mar 4, 2022

shreyapandit requested a review from mufaddal-rohawala March 4, 2022 22:42

navinsoni reviewed Mar 4, 2022

View reviewed changes

Comment thread src/sagemaker/model.py

bring back script assignment, cleanup test

845bda5

fix indents and cleanup

9967091

navinsoni approved these changes Mar 4, 2022

View reviewed changes

mufaddal-rohawala approved these changes Mar 5, 2022

View reviewed changes

ahsan-z-khan changed the title ~~Fixes container env generation for S3 URI and add test for the same~~ fix: container env generation for S3 URI and add test for the same Mar 7, 2022

ahsan-z-khan merged commit d610bfb into aws:dev Mar 7, 2022

navinsoni pushed a commit that referenced this pull request Mar 9, 2022

fix: container env generation for S3 URI and add test for the same (#…

a39b1e7

…2971)

Evan-W-ang added a commit to Evan-W-ang/sagemaker-python-sdk that referenced this pull request Jun 8, 2026

fix: container env generation for S3 URI and add test for the same (a…

806c044

…ws#2971)

Uh oh!

Conversation

shreyapandit commented Mar 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Checklist

General

Tests

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

jeniyat left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

codecov-commenter commented Mar 4, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 4, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 5, 2022

AWS CodeBuild CI Report

Uh oh!

sagemaker-bot commented Mar 5, 2022

AWS CodeBuild CI Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

shreyapandit commented Mar 4, 2022 •

edited

Loading

codecov-commenter commented Mar 4, 2022 •

edited

Loading