Troubleshooting HubSpot Data Migration: Resolving API Limits, Sync Crashes, and Configuration Errors
Comprehensive guide to fixing HubSpot data migration failures, including API rate limit exhaustion, configuration sync crashes, and property mapping errors.
- API Rate Limits (HTTP 429) are the most common cause of migration crashes; implement exponential backoff and bulk endpoints.
- Property mapping mismatches (HTTP 400) often occur due to mismatched internal values for enumeration or date properties.
- HubSpot configuration drifts between sandbox and production environments can lead to silently failing association syncs.
- Quick Fix: Switch from single-object API calls to the HubSpot CRM Bulk API and validate your schema mapping before initializing the sync.
| Method | When to Use | Time | Risk |
|---|---|---|---|
| Implement Bulk API & Backoff | When encountering 429 Too Many Requests or migration script crashes. | 2-4 Hours | Low |
| Schema Audit & Re-mapping | When encountering 400 Bad Request errors indicating invalid property values. | 1-2 Days | Medium |
| Native HubSpot Import Tool | For one-off static data loads under 1M rows where custom scripts are failing. | 1-3 Hours | Low |
| Middleware Sync (e.g., Workato/Tray) | For continuous syncs where custom HubSpot integrations are persistently not working. | 1-2 Weeks | Low |
Understanding the Error: Why HubSpot Migrations Crash
Executing a HubSpot data migration is rarely a simple lift-and-shift operation. When migrating hundreds of thousands of records (Contacts, Companies, Deals, Tickets, and Custom Objects) into HubSpot from legacy CRM systems like Salesforce, Microsoft Dynamics, or custom databases, engineers frequently encounter a combination of API throttling, schema validation failures, and hidden configuration dependencies.
When a HubSpot migration script is "not working" or experiences a "crash," the root cause is almost always visible in the HTTP response headers or payload returned by the HubSpot API. Understanding these underlying mechanics is critical to building a resilient pipeline.
Common Error Signatures
During a migration, your application logs will typically surface one of the following error signatures:
1. The 429 Too Many Requests (Rate Limiting)
{
"status": "error",
"message": "You have reached your daily limit.",
"errorType": "RATE_LIMIT",
"correlationId": "a1b2c3d4-e5f6-7890-1234-56789abcdef0"
}
This indicates you have breached either the 10-second burst limit (typically 100-150 requests per 10 seconds, depending on your HubSpot tier) or the daily API limit (typically 500,000 API calls per day for Enterprise).
2. The 400 Bad Request (Property Validation Failure)
{
"status": "error",
"message": "Property values were not valid: [{\"isValid\":false,\"message\":\"\"John Doe\" was not one of the allowed options: [label: \"Jane Doe\" value: \"jane_doe\"...",\"error\":\"INVALID_OPTION\",\"name\":\"contact_owner\"}]",
"correlationId": "f0e9d8c7-b6a5-4321-0987-654321fedcba"
}
This occurs when your migration attempts to insert a value into a dropdown, radio select, or multiple checkbox property that does not exactly match the internal value defined in the HubSpot configuration.
3. The 207 Multi-Status (Partial Bulk Failure)
When using the Bulk API, a 207 response means the request was successfully processed by the server, but one or more individual records within the batch failed validation. If your migration script does not parse the errors array in a 207 response, data loss will occur silently.
Step 1: Diagnose the Bottleneck
Before modifying your migration code, you must determine exactly where the pipeline is breaking down.
Audit API Usage and Limits
First, check your current API consumption. HubSpot provides a specific endpoint to view your daily limit usage. If your migration is crashing randomly after running successfully for hours, you are likely hitting the daily ceiling.
Navigate to Settings > Account Setup > Integrations > API Key (or Private Apps, which is the modern standard) in the HubSpot UI to view the Call Log. Alternatively, inspect the HTTP headers of any HubSpot API response. HubSpot returns the following headers:
X-HubSpot-RateLimit-Daily: Your total daily allowance.X-HubSpot-RateLimit-Daily-Remaining: Calls remaining today.X-HubSpot-RateLimit-Interval-Milliseconds: The rolling window for burst limits (usually 10000ms).X-HubSpot-RateLimit-Remaining: Calls remaining in the current 10-second window.
Validate HubSpot Configuration Drifts
If you are migrating data into a HubSpot Sandbox and then promoting to Production, ensure the configuration (custom properties, enumeration options, mandatory fields) is identical. A common cause of "HubSpot configuration not working" during migration is a custom property existing in the Sandbox but missing in Production.
Export the property schemas from both environments and diff them using a tool like jq.
Step 2: Implement Resilient Migration Architecture
To permanently resolve HubSpot data migration crashes, your pipeline must be refactored to handle the realities of distributed systems and strict rate limiting.
Tactic 1: Migrate to Bulk Endpoints
The most critical fix for API exhaustion is abandoning single-object endpoints (e.g., POST /crm/v3/objects/contacts) in favor of the Bulk API (e.g., POST /crm/v3/objects/contacts/batch/create).
The Bulk API allows you to send up to 100 records per HTTP request. This immediately reduces your API consumption by a factor of 100.
Crucial Bulk API Considerations:
- Batch Sizing: Always batch your records in chunks of 100 (the maximum allowed).
- Error Handling: You must inspect the response for a
207 Multi-Statuscode. The response body will contain anerrorsarray detailing exactly which records failed and why, while the successful records will be listed in theresultsarray. You must log the failed records to a Dead Letter Queue (DLQ) for manual review or automated retry.
Tactic 2: Implement Exponential Backoff with Jitter
Even with the Bulk API, aggressive multi-threaded migration scripts can still breach the 10-second burst limit. Your HTTP client must be configured to automatically retry 429 Too Many Requests responses using an exponential backoff strategy.
When a 429 is received, the script should pause for a short duration (e.g., 1 second), retry, and if it fails again, double the wait time (2s, 4s, 8s). Adding "jitter" (a small randomized variance to the wait time) prevents the "thundering herd" problem where multiple threads wake up and retry at the exact same millisecond.
Tactic 3: Standardize Data Transformation
To prevent 400 Bad Request validation errors, implement a strict transformation layer before pushing data to HubSpot.
- Dates: HubSpot expects all
dateanddatetimeproperties to be passed as UNIX timestamps in milliseconds, set to midnight UTC. - Enumerations: Map legacy system string values to the exact
internal valueof the HubSpot property, not thelabel. (e.g., mapping "United States" to "united_states"). - Associations: When migrating associated objects (like Deals linked to Contacts), you must first migrate the Contacts and capture their new HubSpot IDs, then migrate the Deals and use the HubSpot CRM Associations API to link them using those IDs.
Step 3: Handling Complex Object Associations
A frequent source of "HubSpot troubleshooting" searches involves associations failing to migrate.
The optimal sequence for a relational data migration is:
- Migrate all parent objects (e.g., Companies). Store the mapping of
Legacy_Company_IDtoHubSpot_Company_IDin a local database (like Redis or PostgreSQL). - Migrate all child objects (e.g., Contacts). Store the
Legacy_Contact_IDtoHubSpot_Contact_IDmapping. - Run an Association Sync script. This script iterates over your legacy relationships, looks up the corresponding
HubSpot_Company_IDandHubSpot_Contact_IDfrom your local database, and issues a batch association request toPOST /crm/v3/associations/{fromObjectType}/{toObjectType}/batch/create.
By splitting the creation and the association into distinct phases, you drastically reduce the complexity of the initial load and make it much easier to restart the migration if a crash occurs.
Frequently Asked Questions
import requests
import time
import logging
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
# Configure robust session with exponential backoff for HubSpot API
def get_hubspot_session(access_token):
session = requests.Session()
session.headers.update({
"Authorization": f"Bearer {access_token}",
"Content-Type": "application/json"
})
# Retry strategy: Catch 429s and 5xx errors, backoff exponentially
retry_strategy = Retry(
total=5,
backoff_factor=2, # Wait 2, 4, 8, 16 seconds between retries
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["HEAD", "GET", "OPTIONS", "POST", "PUT"]
)
adapter = HTTPAdapter(max_retries=retry_strategy)
session.mount("https://", adapter)
return session
# Example Bulk Contact Migration
def migrate_contacts_bulk(session, contacts_batch):
url = "https://api.hubapi.com/crm/v3/objects/contacts/batch/create"
payload = {
"inputs": contacts_batch # Max 100 dicts per batch
}
try:
response = session.post(url, json=payload)
# Handle successful batch
if response.status_code == 201:
logging.info(f"Successfully migrated batch of {len(contacts_batch)}")
return response.json().get('results', [])
# Handle partial failures
elif response.status_code == 207:
data = response.json()
logging.warning(f"Partial failure. {len(data.get('errors', []))} records failed.")
for error in data.get('errors', []):
logging.error(f"Record Failure: {error['message']}")
return data.get('results', [])
else:
logging.error(f"Migration Crash: HTTP {response.status_code} - {response.text}")
return None
except Exception as e:
logging.error(f"Network or Timeout Error during HubSpot sync: {str(e)}")
return NoneError Medic Editorial
Error Medic Editorial is composed of senior Site Reliability Engineers and DevOps practitioners dedicated to solving complex enterprise integration and infrastructure bottlenecks.