GS Caltex Takes SageMaker No-Code Models to Production in Hours

Summary:South Korean energy and chemical company GS Caltex has connected models and Data Wrangler flows validated by process engineers in Amazon SageMaker Canvas to a production architecture spanning SageMaker Pipelines, Model Registry, Amazon EventBridge, and Batch Transform. ML engineers can reuse the validated assets, enforce approval gates, and run scheduled batch inference every 30 minutes without keeping real-time inference endpoints online. The company reports that deployment time fell from roug

---

Core approach: moving no-code model assets into an orchestrated production pipeline

GS Caltex divides model development and production operations across two complementary roles. Process engineers use Amazon SageMaker Canvas to prepare data visually, build models, and evaluate predictions; ML engineers configure production steps, model versions, and schedules in SageMaker Pipelines.

Models validated in Canvas are registered and approved in SageMaker Model Registry. Data Wrangler .flow files and their associated SageMaker Processing configurations preserve the preprocessing logic used during development. The resulting architecture is:

EventBridge trigger → Pipelines orchestration → Processing preprocessing → Batch Transform inference → S3 result assembly

EventBridge starts the pipeline every 30 minutes, Pipelines enforces dependencies, Processing runs the Data Wrangler flow, Batch Transform performs inference with an approved model, and the final results are stored in S3. Pipeline execution states and CloudWatch logs provide operational visibility.

How the five-step inference pipeline works

The reference implementation divides the workflow into five explicit stages:

1. PrepareInput prepares inference data and adds a placeholder target when required.
2. PreprocessNewData executes the exported Data Wrangler .flow through SageMaker Processing.
3. DropTargetColumn removes the placeholder target and CSV header, producing a headerless feature file.
4. BatchPredict runs inference through SageMaker Batch Transform.
5. AssembleResults joins the preprocessed data with prediction outputs.

Dependencies are expressed with depends_on, while S3 locations are passed through step output properties instead of being hard-coded. This creates independently inspectable stages with explicit input and output boundaries.

The most difficult integration issue is the difference between training and inference schemas. Training data includes the target column, while inference data generally does not. The reference implementation inserts a type- and position-compatible placeholder before preprocessing and removes it afterward. This approach is valid only when the target does not affect feature values. Flows that filter rows by the target, derive features from it, or use target encoding require a more rigorous separation of inference-time transformations and learned state.

Reusing preprocessing assets requires state consistency

A Data Wrangler flow captures more than visible transformation steps. It also carries learned state such as scaling and encoding parameters. Production inference must reuse that state instead of fitting it again for every batch; otherwise, the feature distribution can drift from the space used during training.

Migration checks should cover:

• The S3 source URI embedded in the .flow file;
• The Processing container image and its pinned version;
• Output nodes and output names;
• Learned scaling, encoding, and other fitted parameters;
• Column order, data types, and transformed values produced by Canvas and Processing for identical inputs.

Declaring a SageMaker Pipelines parameter does not automatically replace a URI inside the .flow file. A shared fixed input path can therefore cause concurrent executions to overwrite one another. The safer design is to copy and update the flow for each run, isolate intermediate outputs by execution ID, and confirm that the container image is available in every deployment Region.

Approval must be enforced in the execution path

Model Registry records package metadata and approval status, but it does not automatically reject inference jobs that reference an unapproved package. Deployment code should explicitly verify ModelPackageStatus and ModelApprovalStatus, then create the SageMaker model resource from the approved model_package_arn.

The pipeline configuration should also pin the package ARN and model name. If approval is revoked, schedules and deployment configurations must be updated accordingly; a status shown in the console is not an execution-time control. Registration and deployment occur only when a new model version is released. The 30-minute inference workflow reuses the deployed model rather than registering it on every run.

Batch Transform versus real-time endpoints

SageMaker Pipelines provides serverless workflow orchestration, but Processing and Batch Transform jobs still incur compute charges based on instance type, instance count, and runtime. S3, CloudWatch, and other supporting services add further cost.

| Dimension | Real-time inference endpoint | Pipelines + Batch Transform | |---|---|---| | Resource lifecycle | Instances remain online | Compute is released when each job finishes | | Cost drivers | Endpoint uptime and capacity | Job duration, frequency, instance count, and support services | | Best fit | Sustained online requests with consistent low latency | Periodic processing of accumulated data | | Effect of adding a model | More resident capacity and hosting cost | More processing work, runtime, and concurrency pressure |

A 30-minute schedule can trigger as many as 48 executions per day. Cost analysis must therefore include instance startup, preprocessing, postprocessing, retries, and service wait time—not just inference. Batch Transform can eliminate idle endpoint capacity for workloads that tolerate periodic latency. It is usually a poor substitute for sub-second responses because job startup and orchestration introduce delay.

Schedule overlap also requires explicit handling. Dependencies inside one pipeline do not prevent a separate pipeline execution from starting concurrently. The system should inspect active executions, avoid processing the same input window twice, and isolate each run's output paths.

Result assembly is a critical correctness boundary

The reference implementation merges preprocessed data and predictions by row position. That assumes preprocessing does not reorder or discard records. If a flow sorts, filters, deduplicates, or aggregates rows, predictions can be attached to the wrong source records.

Before production deployment, validate:

• Input and output row counts;
• Stable mapping between source records and predictions;
• Output column names, types, and order against the training feature schema;
• Failure behavior that prevents downstream publication after schema or preprocessing errors.

If stable ordering cannot be guaranteed, attach a unique record ID before preprocessing and join results by key rather than file line number.

Reported results and their limitations

GS Caltex reports that deployment time fell from about one week to a few hours, inference runs every 30 minutes, and the architecture produced meaningful cost savings. The main benefit is reduced repetitive integration work: process engineers retain responsibility for model usefulness and domain knowledge, while ML engineers focus on deployment, monitoring, and operational governance.

The timing and cost improvements are company-reported figures. The post does not disclose the number of models, model types, data volumes, absolute costs before and after deployment, or a controlled benchmark. A more reliable internal evaluation would separately measure development, approval, integration, trial-run, and launch time, then compare endpoint hosting costs with the complete cost of the batch workflow.

Deployment recommendations

Start with one validated model and execute the pipeline manually before enabling a schedule. A practical sequence is:

1. Align Canvas and Processing outputs across columns, types, row counts, and transformed values.
2. Verify that the placeholder target does not alter any feature.
3. Pin and version the preprocessing container, .flow file, and model package.
4. Measure end-to-end runtime and total cost using representative data.
5. Test overlapping executions, retries, and duplicate input windows.
6. Establish CloudWatch alerts for failures, delays, and data-quality anomalies.
7. Record the input window, flow version, package ARN, pipeline execution ARN, and output location for every run.

The architecture's main contribution is not a new modeling method. It connects Canvas's usability with Pipelines' orchestration while retaining governance through Model Registry. It does not eliminate model or data drift, but it materially reduces the integration friction between a validated model and a repeatable production workload.

---

Source: AWS Machine Learning Blog (2026-11-13)

— END —

Tags: #SageMaker Canvas Follow #SageMaker Pipelines Follow #MLOps Follow #Batch Inference Follow #GS Caltex Follow