# Get orders data more efficiently
**URL:** https://community.shiphero.com/t/get-orders-data-more-efficiently/515
**Category:** GraphQL API
**Created:** [June 10, 2020, 4:02am UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515 "2020-06-10T04:02:25Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [June 10, 2020, 4:02am UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/1 "2020-06-10T04:02:25Z")
Hi,
**Goal** : Ingest all order related data into our data lake.
**Question** : How can we do this more efficiently considering both time and credit cost? We are also finding that api response time gets slower as we go through more pages of orders.
**Current method** :
2 queries because we found it was faster this way rather than getting order data within date range
> **1. Query for order ID within date range**
>
> {
> orders(updated\_from: “2020-04-20 00:00:00”, updated\_to: “2020-04-20 23:59:59”,) {
> complexity
> request\_id
> data(first: 90, sort: “order\_number”) {
> pageInfo {
> hasNextPage
> hasPreviousPage
> startCursor
> endCursor
> }
> edges {
> node {
> id
> }
> }
> }
> }
> }
> **2. Query for order data using order ID**
>
> {
> order(id: “T3JkZXI6MTA4NDMyODE2”) {
> complexity
> data {
> authorizations {
> transaction\_id
> authorized\_amount
> postauthed\_amount
> refunded\_amount
> card\_type
> date
> }
> line\_items(first: 90, after:"") {
> pageInfo {
> hasNextPage
> hasPreviousPage
> startCursor
> endCursor
> }
> edges {
> node {
> id
> legacy\_id
> sku
> partner\_line\_item\_id
> quantity
> price
> product\_name
> option\_title
> fulfillment\_status
> quantity\_pending\_fulfillment
> quantity\_shipped
> warehouse
> quantity\_allocated
> backorder\_quantity
> custom\_options
> custom\_barcode
> eligible\_for\_return
> customs\_value
> warehouse\_id
> locked\_to\_warehouse\_id
> subtotal
> barcode
> created\_at
> updated\_at
> order\_id
> product {
> id
> legacy\_id
> account\_id
> }
> serial\_numbers {
> id
> legacy\_id
> line\_item\_id
> serial\_number
> scanned
> created\_at
> updated\_at
> }
> }
> }
> }
> rma\_labels {
> id
> legacy\_id
> account\_id
> order\_id
> rma\_id
> shipment\_id
> shipping\_name
> tracking\_number
> status
> carrier
> shipping\_method
> cost
> box\_code
> dimensions {
> weight
> height
> width
> length
> }
> address {
> name
> address1
> address2
> city
> state
> country
> zip
> phone
> }
> paper\_pdf\_location
> thermal\_pdf\_location
> pdf\_location
> image\_location
> delivered
> picked\_up
> refunded
> needs\_refund
> profile
> full\_size\_to\_print
> partner\_fulfillment\_id
> created\_at
> updated\_at
> }
> returns {
> id
> legacy\_id
> account\_id
> order\_id
> partner\_id
> reason
> status
> label\_type
> label\_cost
> cost\_to\_customer
> shipping\_carrier
> shipping\_method
> exchanges {
> id
> legacy\_id
> exchange\_order\_id
> return\_id
> account\_id
> exchange\_order {
> id
> line\_items {
> edges {
> node {
> id
> }
> }
> }
> }
> original\_return {
> id
> }
> }
> dimensions {
> weight
> height
> width
> length
> }
> address {
> name
> address1
> address2
> city
> state
> country
> zip
> phone
> }
> line\_items {
> id
> legacy\_id
> account\_id
> line\_item\_id
> warehouse\_id
> product\_id
> return\_id
> quantity
> condition
> is\_component
> type
> reason
> created\_at
> updated\_at
> line\_item {
> id
> }
> warehouse {
> id
> }
> }
> created\_at
> }
> id
> legacy\_id
> order\_number
> partner\_order\_id
> shop\_name
> fulfillment\_status
> order\_date
> total\_tax
> subtotal
> total\_discounts
> total\_price
> auto\_print\_return\_label
> custom\_invoice\_url
> account\_id
> email
> profile
> gift\_note
> packing\_note
> required\_ship\_date
> shipping\_lines {
> title
> carrier
> method
> price
> }
> tags
> holds {
> fraud\_hold
> address\_hold
> shipping\_method\_hold
> operator\_hold
> payment\_hold
> }
> flagged
> saturday\_delivery
> ignore\_address\_validation\_errors
> priority\_flag
> allocation\_priority
> shipping\_address {
> first\_name
> last\_name
> company
> address1
> address2
> city
> state
> state\_code
> zip
> country
> country\_code
> email
> phone
> }
> billing\_address {
> first\_name
> last\_name
> company
> address1
> address2
> city
> state
> state\_code
> zip
> country
> country\_code
> email
> phone
> }
> shipping\_lines {
> title
> carrier
> method
> price
> }
> tags
> flagged
> saturday\_delivery
> ignore\_address\_validation\_errors
> priority\_flag
> allocation\_priority
> shipments {
> id
> legacy\_id
> order\_id
> user\_id
> warehouse\_id
> pending\_shipment\_id
> address {
> name
> address1
> address2
> city
> state
> country
> zip
> phone
> }
> picked\_up
> needs\_refund
> refunded
> delivered
> shipped\_off\_shiphero
> dropshipment
> created\_date
> shipping\_labels {
> id
> legacy\_id
> account\_id
> }
> warehouse {
> id
> legacy\_id
> }
> order {
> id
> legacy\_id
> }
> line\_items(first: 100) {
> edges {
> node {
> line\_item {
> id
> legacy\_id
> quantity
> }
> }
> }
> }
> }
> holds {
> fraud\_hold
> address\_hold
> shipping\_method\_hold
> operator\_hold
> payment\_hold
> }
> }
> }
> }
Thank you!
---
### Author:  [@Devin\_Mabra](https://community.shiphero.com/u/Devin_Mabra)
#### Post date: [June 11, 2020, 11:03pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/2 "2020-06-11T23:03:21Z")
Hi, would like an answer to this as well. We have several integrations stalled due to the credit limits.
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [June 16, 2020, 7:27pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/3 "2020-06-16T19:27:55Z")
Hi @jwz @Devin_Mabra
So you are making something like this:
```
query {
orders(order_date_from:"2020-03-04",order_date_to:"2020-03-05") {
request_id
complexity
data(first: 10) {
edges {
node {
id
legacy_id
}
}
}
}
}
```
And then with that ID doing:
```
query {
order(id: "117384888") {
request_id
complexity
data {
id
legacy_id
order_number
partner_order_id
shop_name
fulfillment_status
order_date
line_items(first: 5) {
edges {
node {
id
legacy_id
sku
partner_line_item_id
quantity
subtotal
barcode
created_at
updated_at
order_id
}
}
}
}
}
}
```
Is that correct?
In that case why not do it at once with:
```
query {
orders(order_date_from: "2020-03-04", order_date_to: "2020-03-05") {
request_id
complexity
data(first: 10) {
edges {
node {
id
legacy_id
order_number
partner_order_id
shop_name
fulfillment_status
order_date
line_items(first: 5) {
edges {
node {
id
legacy_id
sku
partner_line_item_id
quantity
subtotal
barcode
created_at
updated_at
order_id
}
}
}
}
}
}
}
}
```
If it’s because of credits, then yes it’s a less expensive approach, but I don’t think it might save you much time and you’ll be making two requests instead of one.
Also for sorting credit limit usage you might also include some pause in between requests.
I do think both approaches should work fine
Let me know if it is because you are experiencing a specific issue when making those.
Thanks in advance!
---
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [June 17, 2020, 5:10pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/4 "2020-06-17T17:10:39Z")
Thanks for your response!
We have taken that approach (Get all order ids first then get order detail by ids) due to api response time increasing dramatically when using cursor to go through orders once the order size is bigger than 4-5k. We increased the page size in our calls within quota limits and with this 2 step process, the total amount of calls and time spent was reduced. We have anywhere from ~4k to to ~74k orders per day. It is currently taking us ~1 calendar day to get ~1 week of data, with variance depending on orders per day. We got ~650k of orders from 2020-05-01 to 2020-05-31.
---
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [June 18, 2020, 5:35pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/5 "2020-06-18T17:35:52Z")
Update with example request IDs
**Total order id download** : 4320
**Next cursor** : YXJyYXljb25uZWN0aW9uOjQzMTk=
**Rest API started** :2020-06-17T22:32:37.148-07:00
**Rest API ended** :2020-06-17T22:32:47.453-07:00
10 seconds to return get order ids only
Query
{“query”:"{ orders(updated\_from: “2020-05-29”, updated\_to: “2020-05-30”) { complexity request\_id data(first: 80, after: “YXJyYXljb25uZWN0aW9uOjQzOTk=” ) { pageInfo { hasNextPage hasPreviousPage startCursor endCursor } edges { node { id } } } } } " }
" **request\_id**":“5eeafcaa7726b9dae07b6181”
**When reach 5k:**
**Rest API started** :2020-06-17T22:38:52.733-07:00,
**Rest API ended** :2020-06-17T22:39:04.561-07:00
12 seconds
" **request\_id**":“5eeafdec7df300f2824d1682”
And with number increases it slow down further, **6400** :
**Rest API started** :2020-06-17T22:43:48.536-07:00
**Rest API ended** :2020-06-17T22:44:03.038-07:00
15 seconds
" **request\_id**":“5eeaff14cfa49aee12c902b3”
Thank you!
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [June 18, 2020, 5:48pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/6 "2020-06-18T17:48:26Z")
Thanks for that @jwz !!
I see what you mean. Let me research a bit to see if there is something we could to to optimize that (because as there is lot of orders it might take that time to load all of them and then paginate)
Or if there is some alternative to what you are doing.
I will let you know asap what I find.
Thanks again!
Tom
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [June 29, 2020, 4:09pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/7 "2020-06-29T16:09:15Z")
Hi @jwz
I just wanted to provide a heads up about this filter `updated_at`:
It is only for the order object so it won’t show all cases, such as when a shipment is created but the status doesn’t change (For example, when you partially fulfill an order).
Let me know if this doesn’t change anything and you still be working with this filter.
Thanks again!
Tom
---
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [June 30, 2020, 4:32pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/8 "2020-06-30T16:32:50Z")
Hi @tomasw
Thanks for the heads up!
We are getting shipments and products separately like below so I think we are OK overall for updated data? Please let me know if not.
**Shipments Query**
> {
> shipments(date\_from:“2020-04-20”, date\_to:“2020-04-21”) {
> complexity
> request\_id
> data(first: 15, after: “”) {
> pageInfo {
> hasNextPage
> hasPreviousPage
> startCursor
> endCursor
> }
> edges {
> node {
> id
> legacy\_id
> order\_id
> user\_id
> warehouse\_id
> pending\_shipment\_id
> address {
> name
> address1
> address2
> city
> state
> country
> zip
> phone
> }
> picked\_up
> needs\_refund
> refunded
> delivered
> shipped\_off\_shiphero
> dropshipment
> created\_date
> shipping\_labels {
> id
> legacy\_id
> account\_id
> }
> warehouse {
> id
> legacy\_id
> company\_name
> }
> order {
> id
> legacy\_id
> }
> line\_items(first: 5, after: “”) {
> pageInfo {
> hasNextPage
> hasPreviousPage
> startCursor
> endCursor
> }
> edges {
> node {
> line\_item {
> id
> legacy\_id
> quantity
> }
> }
> }
> }
> }
> }
> }
> }
> }
**Products Query**
> {
> products(updated\_from: “2020-04-20”, updated\_to: “2020-04-21”) {
> data(first: 10, after: “”, sort: “”) {
> pageInfo {
> hasNextPage
> hasPreviousPage
> startCursor
> endCursor
> }
> edges {
> node {
> id
> legacy\_id
> account\_id
> name
> sku
> barcode
> country\_of\_manufacture
> dimensions {
> weight
> height
> width
> length
> }
> tariff\_code
> kit
> kit\_build
> no\_air
> final\_sale
> customs\_value
> customs\_description
> not\_owned
> dropship
> needs\_serial\_number
> thumbnail
> large\_thumbnail
> created\_at
> updated\_at
> tags
> images {
> src
> position
> }
> vendors {
> vendor\_id
> vendor\_sku
> price
> }
> warehouse\_products {
> id
> legacy\_id
> account\_id
> warehouse\_id
> warehouse\_identifier
> price
> value
> value\_currency
> on\_hand
> inventory\_bin
> inventory\_overstock\_bin
> reserve\_inventory
> replenishment\_level
> reorder\_amount
> reorder\_level
> backorder
> allocated
> available
> custom
> customs\_value
> created\_at
> updated\_at
> }
> }
> }
> }
> }
> }
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [July 2, 2020, 12:22pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/9 "2020-07-02T12:22:15Z")
Hi @jwz
As for the product query are you expecting to see updates at inventory level?
The reason I ask this is that updated\_at doesn’t work for inventory updates, but other updates such as Price, Dimensions, etc.
For the orders query + shipments I will have to ask around just to be sure I’m not missing any scenarios either. I will let you know what I can find about it
Thanks again!
Tom
---
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [July 2, 2020, 2:43pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/10 "2020-07-02T14:43:26Z")
Hi @tomasw
Yes, we do want updates on inventory level. What would you recommend?
Thank you!
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [July 3, 2020, 1:07pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/11 "2020-07-03T13:07:56Z")
Hi @jwz
On inventory level you might want to try with the inventory\_changes mutation, you can filter by date and it should return the inventory changes made during that period with their reason
Something like:
```
query {
inventory_changes(date_from: "2020-06-30") {
request_id
complexity
data(first: 25) {
edges {
node {
user_id
account_id
warehouse_id
sku
previous_on_hand
change_in_on_hand
reason
cycle_counted
location_id
created_at
location {
id
}
}
}
}
}
}
```
---
### Author:  [@jwz](https://community.shiphero.com/u/jwz)
#### Post date: [July 7, 2020, 5:58pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/12 "2020-07-07T17:58:23Z")
Hi @tomasw
Thank you!
We got some errors, could you also take a look at these?
> ```
> {
> "message": "Unexpected Error",
> "operation": "inventory_changes",
> "field": "id",
> "request_id": "5f04b2e20fa8bfa5ea730093",
> "code": 22
> },
>
> ```
---
### Author:  [@tomasw](https://community.shiphero.com/u/tomasw)
#### Post date: [July 8, 2020, 12:57pm UTC](https://community.shiphero.com/t/get-orders-data-more-efficiently/515/13 "2020-07-08T12:57:00Z")
Hi @jwz
The locations part may only work for Dynamic Slotting accounts. Your account is Static Slotting (which means you only have 1 location per product) I tested it and if you remove that part it should work
Something like this:
```
{
inventory_changes(date_from: "2020-07-06") {
request_id
complexity
data(first: 25) {
edges {
node {
user_id
account_id
warehouse_id
sku
previous_on_hand
change_in_on_hand
reason
cycle_counted
location_id
created_at
}
}
}
}
}
```
Let me know if that looks better
Thanks again!