Skip to content

sklearn preprocessor output in 'application/x-recordio-protobuf' fails with either import error or version conflict #767

@SarathChandran

Description

@SarathChandran

System Information

  • Framework (e.g. Sklearn) / Algorithm (e.g. Linear-Learner):
  • Framework Version:
  • Python Version: 3.5
  • CPU or GPU: CPU
  • Python SDK Version:
  • Are you using a custom image: No

Describe the problem

When trying to preprocess input data with sklearn framework and save the output as application/x-recordio-protobuf, I am facing import issues with 'sagemaker' module.

ImportError: No module named 'sagemaker'

If i install sagemaker via os.system('pip install sagemaker') in the preprocess script. I still get a version conflict on boto.

**Collecting sagemaker **
Downloading https://files.pythonhosted.org/packages/8f/38/fa369edea6817971e354594b25ccb75d9f8186445417a07f851416d75b47/sagemaker-1.18.16.tar.gz (188kB)

** Collecting boto3>=1.9.64 (from sagemaker) **
Downloading https://files.pythonhosted.org/packages/3f/1b/4adde85d1cbbe8a8f2fef47276daf496328fc8d7c40d7a8d6a67b0eba45b/boto3-1.9.135-py2.py3-none-any.whl (128kB)

** sagemaker_containers._errors.ClientError: (boto3 1.9.51 (/usr/local/lib/python3.5/dist-packages), Requirement.parse('boto3>=1.9.64'), {'sagemaker'}) **

Since i could not get access to the container, i am puzzled about the problem.

Steps to reproduce:
In Example: Inference Pipeline with Scikit-learn and Linear Learner, include 'accept == application/x-recordio-protobuf'

elif accept == 'application/x-recordio-protobuf':
# os.system("pip install sagemaker --upgrade")
# os.system("pip install boto3 --upgrade")
import sagemaker.amazon.common as smac fails to read this import
buf = BytesIO()
print(prediction)
print(prediction.shape)
smac.write_spmatrix_to_sparse_tensor(buf, prediction)
buf.seek(0)
return worker.Response(buf, accept, mimetype=accept)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions