Second Brain: Crafted, Curated, Connected, Compounded on 10月02日 21:00
ClickHouse访问公共S3桶错误及解决方案
index_new5.html
../../../zaker_core/zaker_tpl_static/wap/tpl_guoji1.html

 

本文分析了ClickHouse在访问公共S3桶时出现的InvalidClientTokenId错误,并提供了通过添加NOSIGN参数强制匿名访问的解决方案。

# Problem

ClickHouse fails to read from public S3 buckets with error:

1
InvalidClientTokenId: The security token included in the request is invalid

# Root Cause

ClickHouse automatically tries to use AWS credentials (from environment variables, IAM roles, etc.) even for public buckets. When these credentials are invalid/expired, AWS SDK calls sts:GetCallerIdentity and fails.

# Solution

Force anonymous access by adding NOSIGN parameter:

12345
FROM s3(    's3://bucket/path/file.csv.gz',    NOSIGN,  -- Forces anonymous access    'CSV')

# Rill Example

In your Rill model YAML, update the s3() function call:

 1 2 3 4 5 6 7 8 9101112
# Before (fails with credential errors)FROM s3(    's3://noaa-ghcn-pds/csv.gz/by_year/2025.csv.gz',    'CSV')# After (works with public buckets)FROM s3(    's3://noaa-ghcn-pds/csv.gz/by_year/2025.csv.gz',    NOSIGN,  -- Add this parameter    'CSV')

Complete working example: GitHub - ClickHouse Rill Example

# When This Happens

    You have AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY set in environment
      having these set locally means it also works without NOSIGN
    Credentials are expired/invalidAccessing public S3 buckets that don’t require authenticationWorking across different machines/environments with different AWS setups

# Quick Test

12
# Verify bucket is publicaws s3 ls s3://bucket-name/ --no-sign-request

Remember: Always use NOSIGN when accessing public S3 buckets in ClickHouse to avoid credential validation issues.


Origin: AWS S3
References:
Created 2025-09-05

Fish AI Reader

Fish AI Reader

AI辅助创作,多种专业模板,深度分析,高质量内容生成。从观点提取到深度思考,FishAI为您提供全方位的创作支持。新版本引入自定义参数,让您的创作更加个性化和精准。

FishAI

FishAI

鱼阅,AI 时代的下一个智能信息助手,助你摆脱信息焦虑

联系邮箱 441953276@qq.com

相关标签

ClickHouse S3 错误解决 匿名访问 NOSIGN参数
相关文章