Secure Drupal API Integration Practices

Image

Integrating third-party APIs into your Drupal site is a powerful way to enhance functionality – whether it's pulling in data from an external CRM, syncing products to an e-commerce platform or automating tasks between systems. But as useful as these integrations are, they also come with their own set of challenges:

• How do you make sure your API calls are secure, efficient and reliable?
• How do you handle timeouts or failures gracefully, especially when dealing with third-party services?

In this article, we’re going to walk through the best practices for integrating APIs with Drupal in a way that’s both secure and scalable. We’ll cover everything from structuring your code to handle API calls properly, to keeping sensitive credentials safe and optimizing for performance with retries and timeouts. 

Whether you’re adding a new API connection or fine-tuning an existing one, these tips will help you build integrations that are solid, maintainable and secure.

Integrating a third-party API in Drupal: Architecture and separation of concerns

Service-first structure (don’t call vendors from controllers)

One of the most important things to get right when integrating APIs is separation of concerns. Directly calling third-party APIs from controllers, forms or even other Drupal components can lead to messy, hard-to-maintain code. 

Instead, you should use a service-first structure, where all outbound API calls are encapsulated in a custom service class. This approach offers several key advantages:

• Dependency injection (DI): By creating a service class for your API calls, you can inject it into various parts of your site (like forms, controllers, cron jobs or queue workers) rather than having to call the vendor directly each time. This centralizes your API logic and makes your code more modular.
• Testability: With a dedicated service class, you can easily mock the API interactions during testing, which makes unit tests for your API calls much easier to write.
• Retry logic and observability: You can implement retries, exponential backoff and logging in one place, making it easier to monitor and debug issues with API calls.

By encapsulating all your outbound API calls in a service, you set up your integration for long-term success – keeping your codebase clean organized and testable.

Refresh hygiene

Token refreshes should be centralized and serialized to avoid issues like simultaneous refresh attempts. A few things to keep in mind:

• Centralize token refresh logic: Centralize the logic for refreshing tokens to avoid duplicate refresh requests.
• Serialize refresh attempts: Ensure only one refresh happens at a time to prevent race conditions.
• Fail closed: If there’s a problem with refreshing the token (e.g., scopes drift), fail gracefully and ensure the system doesn’t allow bad tokens to cause further issues.

Drupal tools for external API consumption

HTTP client (Guzzle)

In Drupal, you can use the http_client (which leverages Guzzle) via dependency injection to handle your API calls. This is a more flexible and powerful way of making HTTP requests compared to using raw PHP functions like file_get_contents().

To configure the HTTP client for robust API communication, ensure your client has appropriate connect and read timeouts. This helps avoid hanging requests when the vendor API is unresponsive.

Also, set a sane user-agent string and include any required headers (like authentication or custom headers) globally in your client. This avoids repetition across individual API calls.

Mapping and persistence

Once you receive a response from the API, normalize the vendor's data into value objects or entities that make sense for your Drupal site. This helps decouple your system from changes in the external API and makes the data easier to work with.

Here’s how:

• Idempotent writes: Ensure that API writes are idempotent by using an external ID as a key for updates. This prevents duplication and ensures that retries don’t introduce inconsistent data.
• Conflict handling: Make sure you have logic to handle conflicts, especially when multiple processes may be writing to the same data.
• Persist cursors: For incremental syncing, store cursors (i.e., the current position in a dataset) in Drupal's State API. This allows you to resume syncs from where they left off, rather than re-syncing all data from the beginning.

Long-work policy

Anything that could potentially exceed the web request timeout limit (typically around 120 seconds) should be moved to background workers. In Drupal, the Queue API or Cron are perfect for this.

Use the Queue API for repeatable tasks like syncing data at regular intervals. This ensures that long-running tasks are offloaded to background processes without blocking user interactions.

For scheduled tasks, use Drupal’s cron system to handle periodic work like data syncing or cleanup jobs. And, if you need to run a one-off job (like backfilling missing data), use the Batch API to chunk the work into manageable pieces. This gives users visibility into the progress and prevents timeouts.

What are the best practices for Drupal API security?

When integrating third-party APIs into your Drupal site, security should always be a top priority. If not handled properly, APIs can introduce vulnerabilities that put both your data and your users at risk. 

Here are some best practices to ensure your API integrations remain secure:

Least privilege, short lifetimes

One of the fundamental principles of API security is least privilege. This means you should only request the minimum level of access required for your integration. 

For example, if your API doesn’t need write access, don’t request it. Similarly, when it comes to authentication, short-lived access tokens are ideal. They have a limited lifespan, so even if they’re compromised, the damage is minimized. 

Refresh tokens should also be stored securely – never in your code or configuration files. Use a runtime secrets store to manage them safely.

PII/data handling

When dealing with personally identifiable information (PII), encryption is key. TLS (transport layer security) should be used to encrypt all data in transit, ensuring it can’t be intercepted by malicious actors. 

For data stored at rest, make sure you’re using strong encryption methods to protect sensitive information. Also, be mindful of what you log—avoid storing tokens, passwords or PII in your logs. If logging is necessary, make sure sensitive data is redacted.

Secrets and authentication

When it comes to handling authentication and API credentials, never store them in your code or config files. Hardcoding credentials or placing them in version-controlled files is a security risk. Instead, use a runtime secrets store to manage credentials securely. This ensures that sensitive data, like API keys or OAuth tokens, is never exposed in your code.

To manage secrets securely, store credentials in environment variables (or parameters in Drupal’s settings). This keeps them out of your code and ensures they are only available at runtime.

Pantheon Secrets, for example, makes it easy to manage these secrets safely, reducing the risk of exposure.

Also, inject these secrets into your service class through Drupal's dependency injection system. This makes it easy to access them where needed without hardcoding them anywhere.

If your API uses OAuth2 for authentication, it’s a good practice to use short-lived access tokens along with refresh tokens. Access tokens should be temporary and renewed via the refresh token mechanism.

Use Drupal’s State API to store ephemeral access tokens (not the Configuration API), as tokens typically expire quickly. Ensure the tokens are time-bound (with TTL) and remove them when no longer valid.

Additionally, ensure your logs never contain access tokens. Always redact or avoid logging sensitive information like authentication tokens.

Improving performance and reliability of Drupal API integrations

Handling API timeouts in Drupal

When integrating Drupal with third-party APIs, timeouts are a common issue in production environments. They occur when requests take longer to complete than the system allows, which usually points back to the way the integration is designed.

A frequent cause is running heavy operations – like bulk data imports – inside a web request. These long tasks should be moved out of the request cycle and into background processes using Drupal’s Queue API or scheduled with cron. This ensures the user’s request completes quickly while the real work happens asynchronously.

Timeouts can also come from making too many API calls in a loop. If your code calls the API once for each record, you’ll hit limits fast. Instead, batch requests or sync data in smaller chunks using pagination or cursor-based methods.

External services themselves can also contribute to timeouts if they respond slowly or experience downtime. To handle this, add retry logic with exponential backoff and jitter, which spaces out retries and prevents overloading the service. And because issues often surface silently, monitoring and alerting are critical – track success rates, queue performance and retry patterns so problems can be caught early.

Tools like Pantheon’s Quicksilver can automate notifications and recovery actions in response to these issues.

Retry logic and exponential backoff

To ensure that your API calls are reliable, you should implement retry middleware with exponential backoff and jitter. Here’s what that looks like:

• Exponential backoff and jitter: If an API call fails, you should retry it, but not immediately – wait for progressively longer intervals between retries (exponential backoff). Jitter adds a random element to the backoff time, preventing the system from hammering the API with retries all at once.
• Honor retry-after header: If the API returns a 429 (Too Many Requests) or 503 (Service Unavailable), make sure to respect the Retry-After header, which tells you when it’s safe to retry.
• Timeouts vs. application errors: Differentiate between a timeout error (e.g., server-side issue) and application errors (e.g., invalid input). Timeout errors should trigger retries, but application errors should not.

To further improve debugging and observability, attach a correlation ID to each API request. This allows you to track a specific request across different services, making it easier to troubleshoot when something goes wrong.

Testing, observability and service level objectives (SLOs) for external integrations

Ensuring reliability before production with testing

Testing your external API integrations is crucial to detecting problems before they make it into production. Here are some testing approaches to ensure your integration runs reliably:

• Contract testing with vendor mocks: Contract tests ensure that your integration complies with the vendor’s API specification. This is especially important when vendor APIs change or evolve. By creating mock responses based on expected vendor behavior, you can test how your Drupal integration handles those responses. Maintaining golden responses – predefined, correct API responses – helps you spot any changes to the schema or unexpected errors. These tests simulate real API interactions and help identify schema drift, ensuring compatibility even when the vendor updates its API.
• Negative testing: This ensures that your system behaves gracefully when things go wrong. Test for scenarios like expired tokens, bad HMAC signatures or clock skew in webhooks. These are common failure points and testing for these errors ensures that your system can reject invalid requests or handle authentication failures without crashing.
• Load testing: Load tests simulate real-world traffic and help determine how your API integration performs under stress. Measure key metrics like queue depth and throughput, especially when interacting with rate-limited APIs. This helps ensure your integration can scale to handle traffic spikes without compromising performance.

Observability: Monitoring integration health

Once your API integration is live, observability becomes key for monitoring its health and diagnosing issues. Here’s how to ensure you can track your system’s performance and stability:

• Structured logs with correlation IDs: Structured logs provide a clear, machine-readable format for tracking events in your system. Use logs to record key information like the correlation ID for each API call. The correlation ID helps you trace a request through the entire system, from the initial API call to the final response. This makes troubleshooting easier and faster, as you can follow the request’s journey and identify where it might have failed or been delayed.
• Key metrics: In addition to logs, track the following key metrics to monitor the health of your integration:
  • Error rate: Track the percentage of failed API calls. A sudden spike in error rates can indicate an issue with the external API or your integration.
  • P95 latency: Measure the 95th percentile latency, which tells you how quickly the API responds under normal conditions. It helps you understand how your integration performs in real-world scenarios, not just under ideal conditions.
  • Queue age: Keep an eye on the age of tasks in the queue. If tasks are piling up and not being processed quickly, it could be a sign of resource contention or a failure in your queue system.
  • Sync currency: This metric tracks how up-to-date your system is with external data. For integrations that sync external data, syncing delays or issues may lead to outdated information being displayed to users.
• Webhook failures: For webhooks, it’s important to track whether incoming webhook requests succeed or fail. A failure could indicate issues like invalid payloads or authentication problems.
• Health endpoint: This endpoint provides a real-time status of your system, showing whether key components like the API integration, queue processing and data syncing are working as expected. Health endpoints typically return a red/amber/green status, indicating whether the system is healthy, degraded or experiencing failures. You can use these endpoints to automate checks and trigger alerts when issues arise.

SLOs and alerts

Setting Service Level Objectives (SLOs) and creating alerting mechanisms ensure that your integration delivers the expected level of service. Here’s how to set SLOs and manage alerts:

• User-visible SLOs: SLOs define the acceptable level of service for your users. For example, if your integration syncs CRM data, an SLO might be “CRM updates should be visible within 15 minutes.” This objective sets clear expectations for performance and helps you prioritize improvements or identify issues quickly.
• Alerting on breach risks: Set up alerts for key metrics that indicate your integration is at risk of not meeting its SLOs. Instead of alerting on every failure, focus on breach risks – situations where metrics exceed predefined thresholds. For example:
• Queue age > 12 minutes: If the queue starts backing up and processing tasks too slowly, it could impact your SLOs. Setting up an alert for this ensures you can act before it becomes a serious problem.
• API error rates: If your error rate exceeds a certain threshold, trigger an alert to investigate potential issues with the external API or your system.
• Runbook with triage steps: It’s essential to have a runbook (a guide for handling issues) that outlines the steps for troubleshooting and recovering from common failures. The runbook should include:
  • Quick triage steps for diagnosing the problem.
  • Replay steps for re-processing failed tasks or syncing missed data.
  • Contact information for vendor support or escalation in case issues cannot be resolved internally.

Platform considerations that matter

When integrating third-party APIs with your Drupal site, choosing the right platform is just as important as following best practices for security, performance and scalability. The right platform should provide tools that simplify your integrations and help you maintain high standards for reliability and security. 

Pantheon provides a platform that is built with these key features in mind, making it the ideal environment for Drupal integrations – here’s how:

• Pantheon Secrets, a secure, platform-level secrets management solution. Through the Terminus plugin/SDK, you can manage API keys and tokens at runtime – perfect for keeping your sensitive data safe and ensuring you never need to store secrets in your codebase or config.
• Pantheon’s Quicksilver automation system runs custom scripts on actions like deploys, clones and cache clears. This means you can easily set up post-deploy scripts to run queue jobs (e.g., drush queue:run), seed data or send notifications on failures – keeping your integrations smooth and reliable.
• Pantheon’s Multidev feature allows you to create branch-scoped environments, including code, database and file changes. You can safely test your API integrations in these isolated environments before merging to ensure everything works correctly and doesn’t break your production site.
• Pantheon’s infrastructure includes clear and well-documented timeout policies – 59 seconds for web requests (via Fastly) and 120 seconds for non-web processes. These time limits help you design your integrations properly, ensuring long-running tasks are handled in background workers or queues, rather than directly within the web request.

You can apply the best practices and patterns outlined throughout the article to any platform, but if you want an out-of-the-box solution designed to reduce operational toil while reinforcing reliable, secure API integrations, Pantheon is the platform for you.

Start with Pantheon today to simplify your integration process and ensure your Drupal site is resilient and secure!