Bidders - Troubleshooting BSS uploads

You can use the methods in this topic to diagnose any problems with your segment data upload.

Different Phases of Batch Segment Service

This section describes the phases of upload, so you can understand where problems may be occurring.

Starting - Request upload URL and Job ID

In this phase, the clients request an upload URL and a job ID. This stage times out in 5 minutes. If jobs are stuck in this phase, it indicates that the clients requested the URL but failed to upload anything within the allotted time.

Uploading

In this phase, the clients upload the file to the given URL. It is recommended not to exceed one upload per minute. If the clients have more than 200 jobs waiting to be processed at any given time, they will be prohibited from uploading additional jobs.

Validating and Processing

Once the clients upload the file, processing of the file occurs in the next phase where the validation of segment IDs and user IDs is carried out. If a record contains invalid segment or user IDs, the platform does not process the record. When the clients check the job status, they will be able to view statistics regarding the number of invalid users.

Completion

In this phase, the data of the file is successfully uploaded to the platform, and it is available for targeting.

Possible upload errors

Attempting to upload file larger than 0.5GB

{"response":{"status":"ERROR","error_code":"FILESIZE_LIMIT_EXCEEDED","errors":["Member
            exceeds maximum byte size allowed for a file"]}}

Error code in batch segment upload job

""batch_segment_upload_job": {
      "phase": "error",
      "start_time": "2015-08-13 18:40:32",
      "uploaded_time": null,
      "validated_time": null,
      "completed_time": null,
      "error_code": "uploading-error",
      "time_to_process": "0.00",
      "percent_complete": 0,
      "num_valid": 0,
      "num_invalid_format": 0,
      "num_valid_user": 0,
      "num_invalid_user": 0,
      "num_invalid_segment": 0,
      "num_invalid_timestamp": 0,
      "num_unauth_segment": 0,
      "num_past_expiration": 0,
      "num_inactive_segment": 0,
      "num_other_error": 0,
      "error_log_lines": null,
      "segment_log_lines": null,
      "id": 11661553,
      "job_id": "Pm3oCUf5CSVKIOt4mAqOzdt6K3qInj1431542432",
      "member_id": 958,
      "created_on": "2015-05-13 18:40:32",
      "last_modified": "2015-05-13 18:40:33"
    }

The following errors may happen when:

1. You have reached one of its four upload limits:

   • daily bytes,
   • hourly bytes,
   • daily lines, or
   • hourly lines

   Attempting to exceed daily byte upload limit

   {"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                     exceeds maximum allowed bytes per day"]}}
   

   Attempting to exceed hourly byte upload limit

   {"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                     exceeds maximum allowed bytes per hour"]}}
   

   Attempting to exceed daily lines upload limit

   {"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                     exceeds maximum allowed number of lines per day"]}} 
   

   Attempting to exceed hourly lines upload limit

   {"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                     exceeds maximum allowed number of lines per hour"]}}
   

2. You have canceled the upload.

3. The upload phase exceeds 90 minutes.

   Exceeding maximum time to upload

   {"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Maximum
                     upload time exceeded"]}}
   

Possible processing errors

Invalid format

If the value of the num_invalid_format field is greater than "0", check the values in the error_log_lines field.

In the example below, the num_invalid_format field displays a value of "1", with details provided in the error_log_lines field.

In the error_log_lines field:

• num_invalid_format indicates that there was a problem parsing a line in the uploaded file.
• "failed with an illegal number of fields" indicates that the number of fields in a segment_fields block did not match what was defined in the batch-segment config (for more information, see Initial BSS Account Setup).

In this case, the config expects three fields to be defined in the block: SEG_ID, VALUE, EXPIRATION, but the parser only found two fields - SEG_ID and VALUE, thus showing an error.

num_invalid_format and error_log_lines example

"batch_segment_upload_job": {
phase": "completed",
"error_code": null,
"time_to_process": "0.01",
"percent_complete": 100,
"num_valid": 0,
"num_invalid_format": 1,
"num_valid_user": 0,
"num_invalid_user": 0,
"num_invalid_segment": 0,
"num_invalid_timestamp": 0,
"num_unauth_segment": 0,
"num_past_expiration": 0,
"num_inactive_segment": 0,
"num_other_error": 0,
"error_log_lines": "num_invalid_format-WINDOWSADID-USER-ID;SEG_ID:VALUE~9 failed with an illegal number of fields",
"segment_log_lines": null,
"start_time": "2015-08-13 18:40:32",
"uploaded_time": "2015-08-13 18:42:32",
"validated_time": "2015-08-13 18:42:32",
"completed_time": "2015-08-13 18:42:33",
"id": 123412341234,
"job_id": "Pm3oCUf5CSVKIOt4mAqOzdt6K3qInj1431542432",
"member_id": 958,
"created_on": "2015-08-13 18:40:32",
"last_modified": "2015-08-13 18:42:33"
}

View your file upload history

To see metadata about all your segment file uploads within the last 30 days, make a GET call to the service with your member_id specified in the query string. The JSON response will include an array of batch_segment_upload_job objects.

For more information about the specific fields of the batch_segment_upload_job object, see JSON Fields.

$ curl -b cookies 'https://api.appnexus.com/batch-segment?member_id=456'
{
   "response" : {
      "batch_segment_upload_job" : [
         {
           "phase": "completed",
            "start_time": "2012-05-22 16:48:55",
            "uploaded_time": "2012-05-22 16:48:56",
            "validated_time": "2012-05-22 16:49:01",
            "completed_time": "2012-05-22 16:49:01",
            "error_code": null,
            "time_to_process": "0.04",
            "percent_complete": 100,
            "num_valid": 0,
            "num_invalid_format": 0,
            "num_invalid_user": 2,
            "num_invalid_segment": 0,
            "num_unauth_segment": 1,
            "num_past_expiration": 0,
            "num_inactive_segment": 0,
            "num_other_error": 0,
            "error_log_lines": " \n\nnum_unauth_segment-4013681496264948522;5013:0,5014:1550\nnum_invalid_user-7652266028043224430;5848:0,5849:1440,5850:1440\nnum_invalid_user-8802117132500293405;5851:0,5847:-1",
            "id": 98,
            "job_id": "T1v98eIOlCZndeLGSXD0nrs57L8ES11337705335",
            "member_id": 456,
            "created_on": "2012-05-22 16:48:55",
            "last_modified": "2012-05-22 16:49:01"
         },
         ...
    }
  }
}

JSON fields

Batch segment upload job

When you request the status of your processing job, the system returns a batch_segment_upload_job object (if you are a data provider, this will be an array containing a single object). Depending on which request you're making to the service, it will contain some or all of the following metadata.

Related topics

• Legacy BSS File Format
• Batch Segment Service Best Practices