Python has tons of cool idioms and features that are often overlooked or underutilized. List comprehensions to cut back on the use of unnecessary loops, decorators to wrap functions with annotations, and generator functions are just some that can be applied to working with the DFP API.
Addressing these errors using decorators
CONCURRENT_MODIFICATION and QUOTA_EXCEEDED errors are similar in nature - the requests you’re making are failing, but not necessarily because the data you’re sending over is bad. In the first case, one of the objects you’re trying to modify is being updated elsewhere, but you likely want to try again after pulling down the same set of objects. You could certainly write code that retries your operations in all of your services for each object, but it may get a bit hard to maintain (especially with duplicated code). A much cleaner implementation would be to use a decorator!
The Python wiki has an entry under the decorators section that shows how a generic decorator might work for retrying a call. With a few modifications, we can tailor this to capture the two types of errors that might arise:
import time from functools import wraps RESPONSES_TO_RETRY = ['CONCURRENT_MODIFICATION', 'QUOTA_EXCEEDED'] def retry(tries=4, delay=3, backoff=2): ''' Decorator that implements an exponential backoff for retrying on errors. Args: tries: int number of times to execute the wrapped function before failing delay: int time to delay in seconds before the FIRST retry backoff: int multiplier to extend the initial delay by for each retry ''' def decorated_function_with_retry(func): @wraps(func) def function_to_retry(*args, **kwargs): local_tries, local_delay = tries, delay while local_tries > 1: try: return func(*args, **kwargs) except Exception, e: if [response for response in RESPONSES_TO_RETRY if response in e.fault['faultstring']]: print '%s, Retrying in %d seconds...' % (str(e), local_delay) time.sleep(local_delay) local_tries -= 1 local_delay *= backoff return func(*args, **kwargs) return function_to_retry return decorated_function_with_retry
Say you were making a call to update line items - with large networks, it’s not unlikely that someone might be editing the line item at the same time. Since you’d want to pull down the most recent copy of the line item any time the update fails, you would want to abstract out the update method to include the getLineItemsByStatement call, e.g.,
@retry() def fetch_and_update_line_item(statement): # call to get the line item in question response = line_item_service.getLineItemsByStatement( statement.ToStatement()) updated_line_items =  if 'results' in response: for line_item in response['results']: # Do something with your line items here and add them to # updated_line_items. line_item_service.updateLineItems(updated_line_items)
This would effectively allow you to, in the event of the update failing due to concurrent modification, pull down and update a new copy of the line item. Using the default constructor will retry 4 times with 3, 6, and 12 second delays in between.
To wrap things up, decorators are incredibly useful constructs in Python and are useful for the DFP API for several reasons:
- Your application will be less flaky and less affected by intermittent application issues.
- You’re less likely to run into quota errors.
- This would prevent overwriting other changes (in the case of retrying failed calls on concurrent modification errors).
- You could also use something like this to log errors on your end, which could help reveal poor code health or inefficient processes.
Make use of decorators in your code, and you'll soon be sitting pretty.