OverView DropPoint
S3-compatible file drop-off, with dedicated buckets per organization
OverView DropPoint is an S3-compatible file drop-off point. Each organization has its own buckets, reached with its own credentials.
Files can be uploaded from the OverView web interface, with the AWS CLI, or with an S3 SDK. That compatibility is what makes automated uploads possible, such as a scheduled daily export. S3 here refers to the protocol: DropPoint is not tied to AWS, and files are stored in France, with a French hosting provider.
Overview
DropPoint provides:
- Complete isolation between organizations
- AWS CLI compatibility for easy integration
- Dedicated S3 buckets per connector type
- Hosted in France, with a French hosting provider, not tied to AWS
Generating Credentials
Steps to Generate Your Credentials
- In the OverView application, navigate to Settings > Security > S3 Credentials
- Click the + button to generate new credentials
- Your credentials are displayed:
- Access Key ID: Unique identifier for your account
- Secret Access Key: Secret key (keep it confidential)
- S3 Endpoint:
https://s3.oversoc.com
Credential Security
- The Secret Access Key is only visible at generation time
- Copy and save your credentials immediately
- Never share your credentials with third parties
Choosing a Client
DropPoint speaks the S3 API, so any S3 client works once it is pointed at
https://s3.oversoc.com. These are the ones covered here:
| Client | Platforms | Suited to |
|---|---|---|
| AWS CLI | Windows, macOS, Linux | The recommended starting point: one tool, same commands everywhere, easy to script |
| AWS Tools for PowerShell | Windows | Windows scripting without installing the AWS CLI, using native cmdlets |
| rclone | Windows, macOS, Linux | Mirroring a folder, since it uploads only what changed |
| Python SDK (boto3) | Windows, macOS, Linux | Uploading from an existing script or application |
A graphical client also works for occasional manual uploads. Cyberduck (Windows,
macOS) and WinSCP (Windows) both accept a custom S3 endpoint. Create a profile
of type S3, set the server to s3.oversoc.com, and use your Access Key ID and
Secret Access Key as username and password.
DropPoint reads the bucket from the path, as in
https://s3.oversoc.com/your-bucket/export.csv. The bucket is never part of the
hostname, so virtual hosted addressing (your-bucket.s3.oversoc.com) does not
work. The clients below are configured accordingly, and a graphical client needs
its "path style" option enabled.
AWS CLI Configuration
Install the AWS CLI
# Windows (PowerShell)
winget install --id Amazon.AWSCLI
# macOS
brew install awscli
# Linux
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o awscliv2.zip
unzip awscliv2.zip && sudo ./aws/install
Configure the profile
Configure your AWS CLI profile using the credentials generated above:
aws configure set aws_access_key_id YOUR_ACCESS_KEY --profile ovs_your_org
aws configure set aws_secret_access_key YOUR_SECRET_KEY --profile ovs_your_org
aws configure set region fr-par --profile ovs_your_org
aws configure set endpoint_url https://s3.oversoc.com --profile ovs_your_org
aws configure set s3.addressing_style path --profile ovs_your_org
The same five commands work on Windows, macOS and Linux. s3.addressing_style
defaults to auto, which tries to put the bucket in the hostname: setting it to
path keeps requests in the form DropPoint accepts. endpoint_url in the
profile is what lets the later commands omit --endpoint-url every time. It
requires AWS CLI v2.13 or later; on an older version, pass
--endpoint-url https://s3.oversoc.com on each command instead.
Check that the profile works:
aws s3 ls --profile ovs_your_org
The command lists your organization's buckets. An empty output means the profile reached DropPoint but no bucket exists yet.
Using S3 Buckets
Bucket Structure
Each organization has dedicated S3 buckets for each connector. Buckets are named directly after the connector type:
azure- For Azure datasentinelone- For SentinelOne datavmware- For VMware data- And other connectors based on your needs
Files go at the root of the bucket, in CSV or JSON. OverView detects new files and imports them on the next synchronization. Each connector's page names the bucket it reads.
List Your Available Buckets
# List all available buckets for your organization
aws s3 ls --profile ovs_your_org
Uploading Files
Via AWS CLI
# List contents of a specific bucket
aws s3 ls s3://your-bucket/ --profile ovs_your_org
# Upload a file to the bucket
aws s3 cp file.json s3://your-bucket/file.json --profile ovs_your_org
# Upload a whole folder
aws s3 sync ./exports/ s3://your-bucket/ --profile ovs_your_org
# Download a file from a bucket
aws s3 cp s3://your-bucket/file.json ./file.json --profile ovs_your_org
Keep the listing in a script: it is what tells an unattended run whether the upload landed.
Via AWS Tools for PowerShell (Windows)
The AWS Tools for PowerShell give Windows native cmdlets, with no AWS CLI to install. Install the S3 module once:
Install-Module -Name AWS.Tools.Installer -Repository PSGallery -Scope CurrentUser
Install-AWSToolsModule -Name AWS.Tools.S3 -Scope CurrentUser
Store the credentials as a named profile, then set the region for the session:
Set-AWSCredential -AccessKey YOUR_ACCESS_KEY -SecretKey YOUR_SECRET_KEY -StoreAs ovs_your_org
Set-DefaultAWSRegion -Region fr-par
List the buckets and upload a file:
Get-S3Bucket -ProfileName ovs_your_org -EndpointUrl https://s3.oversoc.com -Region fr-par
Write-S3Object -BucketName your-bucket -Key export.csv -File .\export.csv `
-ProfileName ovs_your_org -EndpointUrl https://s3.oversoc.com -Region fr-par
-Region on every cmdletSet-AWSCredential -StoreAs stores the keys but no region, and the SDK needs
one to sign a request. Set-DefaultAWSRegion covers the current session, while
-Region on each cmdlet also covers a script run from Task Scheduler, where no
session default has been set.
Path style addressing needs no extra flag: the cmdlets switch to it on their own
as soon as -EndpointUrl is supplied.
Via rclone
rclone compares source and destination, so a scheduled run uploads only what
changed. Add a remote to rclone.conf:
[droppoint]
type = s3
provider = Other
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
endpoint = https://s3.oversoc.com
region = fr-par
force_path_style = true
force_path_style is not optional here, for the reason given above.
The config file lives at %APPDATA%\rclone\rclone.conf on Windows and
~/.config/rclone/rclone.conf on macOS and Linux. rclone config file prints
the path in use.
# List the buckets
rclone lsd droppoint:
# Copy one file
rclone copy ./export.csv droppoint:your-bucket/
# Mirror a folder, uploading only what changed
rclone sync ./exports/ droppoint:your-bucket/
sync deletesrclone sync makes the destination match the source, which means removing
files from the bucket that no longer exist locally. Use rclone copy to add
without ever deleting, and try a run with --dry-run first.
Via Python SDK (boto3)
import boto3
# Configure S3 client
session = boto3.Session(
aws_access_key_id='YOUR_ACCESS_KEY',
aws_secret_access_key='YOUR_SECRET_KEY',
region_name='fr-par'
)
s3_client = session.client(
's3',
endpoint_url='https://s3.oversoc.com'
)
# List all available buckets
response = s3_client.list_buckets()
for bucket in response['Buckets']:
print(f"Bucket: {bucket['Name']}")
# Upload a file to the Azure bucket
s3_client.upload_file(
'local_file.json',
'your-bucket',
'remote_file.json'
)
# List objects in a specific bucket
response = s3_client.list_objects_v2(Bucket='your-bucket')
for obj in response.get('Contents', []):
print(f"File: {obj['Key']}, Size: {obj['Size']} bytes")
Automating Uploads
Pair an upload with a scheduled task so exports reach OverView without anyone logging in. Each upload triggers a new import, so a daily export keeps the inventory current.
Replace your-bucket with the connector's bucket name, shown on that
connector's own documentation page.
Linux
# /etc/cron.d/overview-export : every day at 02:30
30 2 * * * root /usr/local/bin/aws s3 cp /var/exports/export.csv s3://your-bucket/export.csv --profile ovs_your_org
cron runs with a minimal environment, so give aws its full path. The profile
is read from the home directory of the user the job runs as, root above.
macOS
cron works the same way through crontab -e. launchd is the supported
scheduler on macOS and survives reboots more predictably:
<!-- ~/Library/LaunchAgents/com.oversoc.export.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<plist version="1.0"><dict>
<key>Label</key><string>com.oversoc.export</string>
<key>ProgramArguments</key>
<array>
<string>/opt/homebrew/bin/aws</string><string>s3</string><string>cp</string>
<string>/Users/you/exports/export.csv</string>
<string>s3://your-bucket/export.csv</string>
<string>--profile</string><string>ovs_your_org</string>
</array>
<key>StartCalendarInterval</key><dict><key>Hour</key><integer>2</integer><key>Minute</key><integer>30</integer></dict>
</dict></plist>
Load it with launchctl load ~/Library/LaunchAgents/com.oversoc.export.plist.
Windows
Save the upload as a script, so the scheduled task holds one command:
# C:\Scripts\Send-OverViewExport.ps1
$ErrorActionPreference = 'Stop'
Write-S3Object -BucketName your-bucket -Key export.csv -File 'C:\Exports\export.csv' `
-ProfileName ovs_your_org -EndpointUrl https://s3.oversoc.com -Region fr-par
Register it with Task Scheduler:
$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
-Argument '-NoProfile -ExecutionPolicy Bypass -File C:\Scripts\Send-OverViewExport.ps1'
$trigger = New-ScheduledTaskTrigger -Daily -At 02:30
Register-ScheduledTask -TaskName 'OverView export' -Action $action -Trigger $trigger
The task runs as the registered user, and Set-AWSCredential stores the profile
per user. Register the task under the same account that stored the credentials,
or the script will not find the profile.
Troubleshooting
Common Errors
Every client surfaces the error code DropPoint returns. The code is what tells the causes apart:
| Error | What it means | What to check |
|---|---|---|
InvalidAccessKeyId | The Access Key ID is unknown | A typo in the key, or credentials that were revoked. Generate new ones from Settings > Security > S3 Credentials |
SignatureDoesNotMatch | The signature does not match | Usually a wrong Secret Access Key. Also happens when the machine clock is off by more than about 15 minutes, since the signature covers the request timestamp: check NTP |
AccessDenied | Authenticated, but not allowed here | The bucket is a connector these credentials do not cover. Confirm the bucket name against the connector's page, then contact OverSOC support to have the connector added to your credentials |
AccessDenied, "Credentials are inactive" | The credentials were disabled | Generate new credentials, or contact OverSOC support |
AccessDenied, "Missing authorization" | The request carried no signature | The client is not configured for S3 signature v4, or is sending anonymous requests |
EntityTooLarge | One request went over 500 MB | Upload larger files in parts. aws s3 cp and rclone do this on their own; a hand-written SDK call may need multipart enabling. Splitting the export also works |
NoSuchKey | The object does not exist | The key is spelled differently, or the file went into another bucket |
InvalidBucketName | The bucket name is not accepted | The name must be the connector identifier, lowercase, as shown on the connector's page |
XAmzContentSHA256Mismatch | The body does not match its checksum | A proxy is altering the request. Exclude s3.oversoc.com from any rewriting or inspecting proxy |
Where to read the error
# The CLI prints the code on failure; --debug shows the full exchange
aws s3 cp ./export.csv s3://your-bucket/export.csv --profile ovs_your_org --debug
Support
For assistance, contact support@oversoc.com.
Limits and Quotas
- Maximum size per request: 500 MB. A larger file still goes through, in
parts:
aws s3 cpandrcloneswitch to a multipart upload on their own, and each part stays under that limit. Over it, a single request is rejected withEntityTooLarge. - Number of files and total volume: no per-organization quota is enforced today.