Is the API blinking in and out?

Hello!

I’m getting a lot of “Service not available” (code 21) and “Unexpected error” (code 22) errors in queries that normally give me no errors. This has been going on since at least 3:00pm EDT today, July 30.

Here are some request IDs I got back when trying to use the picks_per_day query:

5f231e2a88423e6591ebe6cb (with the code 21)
5f231d024ddb2e80aa367089 (with the code 22)

The ShipHero Status page shows that everything should be operational, but I can’t get back any results.

Thanks!

(sorry to tag you right away, @tomasw, but if something is down, I just want to make sure it gets seen quickly.)

Hi,

We have been having the same issues since ~11:30am PT as well.

5f2313604ddb2e80aa364d80
5f2313c74ddb2e80aa364f48

Thanks also!

Me as well. Getting code 22 on my end doing the simplest of queries. Can you provide an update on what is going on or what has changed?

Hi @fairweather @jwz @jeremyw
I apologize for that, the API should be back to normal now.
We had an issue with one of our recent deploys but it has been resolved now.
Please let me know if there is anything else I could help with.
Thanks for the patience
Tom

1 Like

Did allocation_priority go back to a bool?

Right now it is as Boolean @fairweather
I apologize for that, the rollback we had to do about that last deploy made this change to go back to Boolean.
We are currently working on fixing this so we can get the change back On.
I will update as soon as I have more information about it.
Thanks again!
Tom

Hi @fairweather!
I just wanted to let you know that the field for allocation_priority should now be an Int again
Thanks again for the patience while we worked on it
Tom

Thanks. I figured you must have pushed the change when my integration broke and the error messages started pouring in. Is there a notification system where we can know exactly when these changes are happening?

I apologize for that @fairweather
Changes should not be like this one anymore. Our practice is to deprecate the field and create a new one, that way we don’t impact existing integrations. This case was an exception and we do not intend to make it this way again.
On the other hand, communications are handled by announcements on community and over email (depending on how impacting they are)
We are working on improving our communication skills so announcements are done as expected through community announcements
Would you rather have an mailing list to subscribe to? Maybe I could make that possible
Thanks again for the patience!
Tom

Hi @tomasw,

Ever since I first posted this almost 2 weeks ago, I’m still running into frequent incidents where the API will spin for 60 seconds and then return with an error code 22 “Unexpected error”. I’m running into it more with the picks_per_day and packs_per_day queries when filtering with a date_from. Below are several request IDs from the last 10 minutes (between about 8:35pm EST and 8:45pm EST on Tue, Aug 11, 2020), but I run into this all through the day. Often times I will get one or two failures, and then the same unaltered query will return a valid result on subsequent tries. It’s making data gathering very painful, and testing of new code nearly impossible.

packs_per_day:
5f333b0fa01d03ec1889c78d

picks_per_day:
5f32f3f0d16d451af7201738
5f3339c1d973260123132ff7
5f333a3c566216d4eb40b2b4
5f333a89accf2d936aaab791

Thanks,
Jeremy

Hi @jeremyw
Thanks for those examples. I opened an investigation about this.
For those request ids you get the error, right?
Thanks again!
Tom

Hi @tomasw,

Correct. Those are all ones where I got error code 22.

Thanks!

Hi @tomasw,

Has there been any update on this? Here’s another failed (code 22) request id: 5f3d0a26c10e28be300fd976

Thanks,
Jeremy

Hi @tomasw,

I’m still looking for updates on the connection issues with the API. I’m still frequently getting code 22 “Unexpected Error” when running queries, but sending the query again on subsequent tries (even without changing the query) will sometimes yield results, although it may take several attempts. Part of the big issue with this is that for each of these code 22 returns, it takes a full minute for the result to come back, making the run time a lot longer than when it successfully gives results.

Here are more request_ids for queries that are coming back code 22:

5f4402bb1537ca9bdeefa8de
5f44135c20162f5eee8f0652
5f4413a6bfa5196c9c0df9f3
5f44151e863baae7169787f7
5f441570bfa5196c9c0e001b

Thanks,
Jeremy

I ran across something that my be a useful piece of information. When I run these queries (in the case of the last set of code 22 responses, the packs_per_day query) I’ve been using the date_from criteria (set to the last day or two) and using first:1 to get the first record. Those are the ones where I often get the code 22 errors. But I found that if I specify an order number in the criteria, not only does it seem to work every time, but it seems to come back much faster. Querying without an order number, even when returning a single record, will sometimes take 45 seconds or so when it doesn’t return an error. Specifying an order number takes a second or less to get the same record.

Unfortunately, I cannot specify orders for every packing record I need to pull, but seeing your response in this other thread about the possibility of nonexistent SKUs made me think something like that may be going on.

Thanks,
Jeremy

Hi @jeremyw!
I apologize for the delayed response. Our engineers are still working on this issue. I will post here as soon as they have an update about it

On the other hand, if you use the user_id filter does it improve speed?

Something like this:

query {
  packs_per_day(
	user_id:"VXNlcjoxNzMzNJ=="
    date_from: "2020-08-24"
  ) {
    request_id
    complexity
    data {
      edges {
        node {
          id
          shipment_id
          warehouse_id
          total_items
          unique_items
          barcodes_scanned
          created_at
          order {
            id
            order_number
          }
        }
        cursor
      }
    }
  }
}

You could get the user_id using the legacy_id with this Query

query{
  uuid(legacy_id: 9123, entity: User){
    request_id
    complexity
    data {
      id
    }
  }
}

Thanks again!
Tom

Hi @tomasw,

Specifying the user_id didn’t seem to make much difference. But doing that wouldn’t be a good long term solution anyway because I need to be able to pull data for all users, and from what I can tell, there isn’t a query that will give me a full list of users in the system. I would have to maintain that manually outside of the system and scripts.

Has there been any update on the issue otherwise? I’m still running into the code 22 Unexpected Errors.

Here are 3 failed request_ids from around 3:20PM EDT ON 9/9/2020:
5f5929668ad53b4bb26e0725
5f5929a637f0febc949108a3
5f5929e5311c485fb61d7f90

This fourth request_id was the exact same query that went through a minute or two later:
5f592ad7311c485fb61d7fed

Thanks,
Jeremy

Hi @jeremyw ur engineering team was working on improving the performance lately, so I just submitted those new examples and asked for an update.
I will let you know as soon as I have more information about this.
Thanks again for the patience!
Tom

Thanks, @tomasw. I just wanted to include any new information just in case any differences from one to the next helps find a pattern. :slight_smile:

Jeremy

Hello, @tomasw!

I’ve been having a heck of a time this morning with getting the code 22 Unexpected Errors this morning. My query for downloading picks_per_day keeps timing out and giving me the error. The request_ids from my failed attempts so far this morning (from about 8am EST through current, which is about 8:35am EST) follow:

5fda072e2ee0102c86bfef63 (timed out on the first try)
5fda08925b7621275dcfcc7d (got through 57 pages, or 5700 picks, before it suddenly timed out again)
5fda0aa904decac099995659 (got only 1 page and then timed out again)
5fda0c5cfb7dfee8707d079f (got through 3 pages and then timed out again)

Thanks,
Jeremy