serverless.yml for Python: Packaging, Dependencies, and the Size Limit You Will Hit

A serverless.yml for Python needs a provider block naming the runtime, a functions block mapping handlers to events, and — the part the quickstarts gloss over — a plan for getting your dependencies into the deployment package.

The configuration is straightforward until you import something that is not in the standard library. That is where most of the friction lives, and where the two limits that shape everything else start to bite: 50MB zipped for a direct upload, 250MB unzipped once it is running.

Table of contents

• The minimal file
• Dependencies
• The size limits
• Stages, secrets, and IAM
• Local development
• Whether this is the right shape at all
• How this fits the rest of the stack
• FAQ

The minimal file

service: order-api

provider:
  name: aws
  runtime: python3.12
  region: eu-west-1
  memorySize: 512
  timeout: 20
  environment:
    TABLE_NAME: ${self:service}-${sls:stage}
    LOG_LEVEL: INFO

functions:
  create_order:
    handler: src/orders.create
    events:
      - httpApi:
          path: /orders
          method: post

  process_queue:
    handler: src/worker.handle
    timeout: 300
    events:
      - sqs:
          arn: !GetAtt OrderQueue.Arn
          batchSize: 10

handler: src/orders.create means the create function in src/orders.py. Path separators for the directory, a dot for the function — mixing those up produces an import error at invocation time rather than at deploy time.

Note timeout: 300 on the queue worker overriding the provider default of 20. Set the timeout per function; a value that suits an HTTP handler will kill a batch job, and a value that suits a batch job leaves a broken HTTP handler hanging for five minutes.

memorySize also controls CPU allocation, which is the thing people miss. A CPU-bound function at 256MB can be slower and more expensive than the same function at 1024MB, because it runs more than four times as long. Worth measuring rather than assuming lower is cheaper.

Dependencies

Lambda ships the standard library and the AWS SDK. Everything else you bring yourself.

npm install --save-dev serverless-python-requirements

plugins:
  - serverless-python-requirements

custom:
  pythonRequirements:
    dockerizePip: true
    slim: true
    strip: false

Three settings doing real work.

dockerizePip: true builds dependencies inside a container matching the Lambda environment. Any package with compiled extensions — pydantic, cryptography, psycopg2, numpy, pillow — must be built for Amazon Linux. Build them on a Mac and you get a wheel that imports fine locally and fails in Lambda with an error about an invalid ELF header. This flag is the fix, and it requires Docker running locally.

slim: true strips .pyc files, tests, and documentation from the packaged dependencies. It routinely cuts package size by a third or more, and there is no downside.

strip: false disables stripping symbols from shared objects. Leave it false — stripping breaks numpy and several scientific packages in ways that produce confusing runtime errors.

For multiple functions with different dependency sets, package them separately so each gets only what it imports:

package:
  individually: true
  patterns:
    - "!**"
    - "src/**"
    - "!**/__pycache__/**"
    - "!**/tests/**"

The size limits

Two ceilings, and both are hard:

• 50MB zipped for a direct upload. Larger packages must go via S3, which the framework handles automatically.
• 250MB unzipped, including layers. This one has no workaround within the zip deployment model.

Getting near the second limit is common with data or ML dependencies — pandas alone is around 60MB unzipped, and adding numpy and scipy pushes past 250MB before you have written any code.

The options, roughly in order of how much they cost you:

Exclude what you do not need. Test fixtures, notebooks, sample data, and the boto3 you never needed to bundle in the first place — Lambda already provides it.

custom:
  pythonRequirements:
    noDeploy:
      - boto3
      - botocore
      - pytest

Use a layer for large shared dependencies. AWS publishes a managed layer for pandas and numpy, and layers are shared across functions rather than duplicated per function. They still count toward the 250MB unzipped limit.

Switch to container images, which raise the limit to 10GB. This is the honest answer for anything genuinely large. You write a Dockerfile instead of relying on the packaging plugin, cold starts get slower, and the size problem goes away.

Reconsider the architecture. A function that needs 200MB of scientific libraries is frequently a long-running service wearing a Lambda costume, and it will also be fighting the 15-minute execution ceiling and cold starts proportional to the package size.

Stages, secrets, and IAM

provider:
  stage: ${opt:stage, 'dev'}
  environment:
    STAGE: ${self:provider.stage}
    DB_PASSWORD: ${ssm:/order-api/${self:provider.stage}/db-password}

  iam:
    role:
      statements:
        - Effect: Allow
          Action:
            - dynamodb:GetItem
            - dynamodb:PutItem
          Resource: !GetAtt OrdersTable.Arn

${ssm:...} resolves from Parameter Store at deploy time, so credentials are not in the repository. Note that the resolved value is then stored as a plaintext Lambda environment variable, visible to anyone with console access — for genuinely sensitive material, fetch it from Secrets Manager at runtime instead.

Scope the IAM statements to specific resources. The default when people are in a hurry is Resource: "*", which grants every function in the service full access to every table in the account. It removes the permissions error and creates a much larger one.

Local development

plugins:
  - serverless-python-requirements
  - serverless-offline

npx serverless offline
# then: curl -X POST localhost:3000/dev/orders -d '{"sku":"A-1"}'

serverless-offline emulates API Gateway locally. It is close enough for handler logic and not close enough for IAM, VPC networking, or cold start behaviour, so it is a development convenience rather than a substitute for deploying to a real stage.

Invoke a deployed function directly when you need the real environment:

npx serverless invoke -f create_order --stage dev --log

--log returns the CloudWatch output with the response, which saves a trip to the console.

Whether this is the right shape at all

Serverless earns its place for spiky, event-driven work: a webhook receiver that gets nothing for hours then a thousand requests in a minute, a scheduled job, an image resize triggered by an upload. Scale-to-zero is genuinely valuable when the baseline load is close to zero.

It earns it less for a standard web application with steady traffic. You take on cold starts, a 15-minute execution ceiling, awkward database connection pooling — every invocation is a separate execution context, which is why Lambda plus a relational database ends up needing a connection proxy — and a local development story that never quite matches production.

And the packaging work above is not nothing. The plugin, the Docker requirement, the size limits, and the per-function packaging config are all overhead that exists because the deployment model is unusual.

For an application that is up all the time, a long-running service is simpler in every dimension that matters. Deploying that on RunxBuild is pushing a repository: the build runs on the platform, dependencies install normally with no size ceiling and no cross-compilation, the service gets a live route, connections to a managed database stay open the way the driver expects, and the runtime logs are per deploy. No serverless.yml, no layer arithmetic, no cold starts.

Use functions for the events. Use a service for the application. The mistake is picking one model for both because the first thing you built happened to fit it.

How this fits the rest of the stack

Per-request pricing is hard to reason about until the invoice arrives, which is why comparing it against a predictable runtime is worth doing early. The RunxBuild hosting calculator puts compute, database, storage, and bandwidth on one page so the comparison is against a real number.

Useful related references:

• Python Not Equal: != vs is not, and Why the Difference Bites
• Python Integer Division: Why // Floors and Why -7 // 2 Is -4
• Python for Websites: Where It Fits and Where It Does Not
• Python services on RunxBuild

FAQ

How do I add Python dependencies to a serverless.yml?

Install the serverless-python-requirements plugin, list it under plugins, and keep a requirements.txt. Set dockerizePip: true so packages with compiled extensions build against the Lambda environment, and slim: true to strip bytecode and docs from the package.

Why does my Lambda fail with an invalid ELF header?

A dependency with compiled extensions was built on your machine rather than for Amazon Linux — common with cryptography, psycopg2, numpy, and pillow. Set dockerizePip: true in the pythonRequirements config, which builds them inside a matching container. Docker must be running locally.

What are the Lambda deployment package size limits?

50MB zipped for a direct upload and 250MB unzipped including layers. Data and ML dependencies hit the second limit quickly — pandas alone is roughly 60MB unzipped. Container images raise it to 10GB if you genuinely need the space.

How do I keep secrets out of serverless.yml?

Use ${ssm:/path/to/param} to resolve from Parameter Store at deploy time rather than committing values. Note that the resolved value becomes a plaintext Lambda environment variable, so genuinely sensitive material is better fetched from Secrets Manager at runtime.

Should I use serverless for a normal web application?

Usually not. Serverless suits spiky event-driven work where scale-to-zero matters. A steady-traffic application inherits cold starts, a 15-minute execution ceiling, awkward database connection pooling that often needs a proxy, and the packaging overhead described above. A long-running service is simpler for that case.