how to run a batch transform job in sagemaker pipeline via custom inference code? - amazon-sagemaker

Question

based on the aws documentation/example provided here , https://sagemaker-examples.readthedocs.io/en/latest/sagemaker-pipelines/tabular/abalone_build_train_deploy/sagemaker-pipelines-preprocess-train-evaluate-batch-transform.html#Define-a-Transform-Step-to-Perform-Batch-Transformation, a model is created and a batch transform inference can be run on the trained model. it works for this example but if we need a custom inference script, How do we include a custom inference script in the model or model package before we run the batch transformation ?
from sagemaker.transformer import Transformer
from sagemaker.inputs import TransformInput
from sagemaker.workflow.steps import TransformStep
transformer = Transformer(
model_name=step_create_model.properties.ModelName,
instance_type="ml.m5.xlarge",
instance_count=1,
output_path=f"s3://{default_bucket}/AbaloneTransform",
)
step_transform = TransformStep(
name="AbaloneTransform", transformer=transformer, inputs=TransformInput(data=batch_data)
)

Answer 1

You need a "model repacking step".
From the Amazon SageMaker Workflows FAQ:
Model repacking happens when the pipeline needs to include a custom
script in the compressed model file (model.tar.gz) to be uploaded to
Amazon S3 and used to deploy a model to a SageMaker endpoint. When
SageMaker pipeline trains a model and registers it to the model
registry, it introduces a repack step if the trained model output from
the training job needs to include a custom inference script. The
repack step uncompresses the model, adds a new script, and
recompresses the model. Running the pipeline adds the repack step as a
training job.
Basically, you can redefine a sagemaker Model by calling the training output as model_data and pass the inference script as entry_point.
Then sequentially, after training the model, redefine the model by changing the entry_point and on the latter you can use the transformer.
This is an example flow taken from a tested code:
my_model = Model(
image_uri=your_img_uri,
model_data=step_train.properties.ModelArtifacts.S3ModelArtifacts,
role=role,
entry_point='inference_script.py',
name="your_inference_step_name"
)
step_create_model = ModelStep(
name="YourInfName",
step_args=my_model.create(instance_type="ml.m5.xlarge")
)
transformer = Transformer(
model_name=step_create_model.properties.ModelName,
instance_count=your_instance_count,
instance_type=your_instance_type,
output_path=your_path
)
Of course, instead of the generic Model, you can directly use the one that best suits your requirements (e.g. PyTorchModel etc...).