Almost everyone who sets out to "use the Instagram API" for research, influencer discovery or competitor tracking hits the same wall in the first hour: the official Graph API does not do what they assumed it did. This guide is the map I wish I'd had — what the official API really gives you, where the public/private line actually falls, how the rate limiting behaves once you start pulling at volume, and why profile, hashtag and post scraping are three different problems wearing the same hoodie.
The Graph API's real boundary
The Instagram Graph API is built for businesses managing their own presence, not for reading the platform. Three hard facts shape everything:
- It only sees accounts you administer. To read a profile's media and insights through Graph, that account must be a Business or Creator account and connected to a Facebook Page you have admin rights on. Your competitor's account is not on that list.
- Business Discovery is a thin exception. The
business_discoveryfield lets you look up other Business/Creator accounts by exact username and read public-ish fields: follower count, media count, and recent media with caption, like and comment totals. It will not return personal accounts, follower lists, or Stories. - Hashtag search is rationed. The Hashtag Search API exists but caps you at roughly 30 unique hashtags per 7-day rolling window per user, and only returns top/recent media for Business accounts. It is unusable for broad trend mining.
So if your job is "tell me about accounts and content I don't own," the Graph API answers "I can't" for the majority of it. That is by design, not a bug you can permission your way out of.
Public vs login-walled: where the line really is
Instagram serves a lot through unauthenticated internal JSON endpoints (the ones the web client itself calls), but the surface shrinks every quarter. Roughly:
| Data | Readable without login? | Notes |
|---|---|---|
| Bio, follower / following counts, post count | Usually yes (public accounts) | Served via the profile JSON endpoint; first page of grid included. |
| Recent grid posts + captions + like/comment totals | Usually yes | Pagination beyond the first page often forces a session. |
| Full comment threads on large posts | No | Gated after the first few comments. |
| Follower / following lists | No | Authenticated only, and the most aggressively rate-limited call on the platform. |
| Stories, Highlights, Reels insights | No | Session required; insights only for owned accounts. |
| Hashtag feeds (top + recent) | Rarely without a session | Heavily gated; this is where most scrapers break first. |
HTTP 200 does not mean you got data. Instagram frequently answers a flagged request with a 200 whose body is a login-redirect page or an empty {"data": {"user": null}}. Always assert on the shape of the payload, not the status code.How the rate limiting actually behaves
There is no published "X requests per minute" number for the web endpoints, because the throttle is reputation-based, not a fixed counter. From running this at volume, the pattern is:
- Velocity + IP reputation drive the block. A clean residential IP pacing one profile every few seconds can run for a long time. A datacenter IP firing 30+ lookups a minute typically gets soft-blocked within a few hundred requests.
- Blocks are soft, not loud. You'll see
429 Too Many Requests, or acheckpoint_requiredredirect, or that empty-200 described above. The session isn't banned forever — it's parked. - Follower-list and hashtag calls burn budget fastest. Treat them as 5–10x more "expensive" than a plain profile read when you plan concurrency.
The practical mitigations are residential proxy rotation, randomized pacing, session pools, and exponential backoff on the soft-block signatures. Building and maintaining that stack is the actual work — the parsing is the easy 10%.
Three scrape types, three different problems
Profile scraping
The most reliable. You pass usernames, you get account-level fields plus the first page of the grid. Good for influencer vetting (followers, engagement rate from recent posts) and lead qualification from bios.
Hashtag scraping
The most fragile. Hashtag feeds are split into "top" and "recent," both gated, both quick to throttle. If a scraper is going to fail on you, it fails here first. Expect lower throughput and plan retries.
Post scraping
Middle ground. A single post's media, caption and counts read cleanly; the comment thread underneath needs a session and paginates slowly. Great for campaign tracking and UGC monitoring, less so for full comment-sentiment at scale.
A pragmatic shortcut: a managed Actor
If you don't want to own the proxy/session/backoff machinery, the no-code route is to call a maintained Instagram scraper on Apify. You hand it usernames, hashtags or post URLs; it returns clean JSON over a REST endpoint and absorbs the rate-limit fight for you. Here's the shape of a synchronous run:
curl -X POST "https://api.apify.com/v2/acts/YOUR_ACTOR_ID/run-sync-get-dataset-items?token=YOUR_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"usernames": ["nike","adidas"],
"resultsType": "posts",
"resultsLimit": 100,
"addParentData": true
}'The response is an array of post or profile objects with fields like ownerUsername, caption, likesCount, commentsCount, timestamp and url — ready to drop into a notebook, a sheet, or your own enrichment pipeline.
Want a ready-made Instagram Actor?
Our maintained Instagram scraper returns profiles, posts and hashtag feeds as clean JSON, handles the proxies and rate limits, and is callable as an API. Free to start with Apify's monthly platform credits.
Get the Instagram Scraper → Or get done-for-you leadsFAQ
Can the Instagram Graph API read any public profile?
No. It only returns data for Business/Creator accounts you've connected via a Facebook Page you administer, plus limited Business Discovery lookups of other Business/Creator accounts by exact username. Personal profiles, follower lists and non-business hashtag feeds are out of reach.
What Instagram data still requires a login?
Follower/following lists, who liked a post, full comment threads, Stories, and most hashtag results. Basic public-account fields and recent grid posts are usually readable unauthenticated, through endpoints that change often.
How do the rate limits work?
They're reputation- and velocity-based, not a fixed published number. Datacenter IPs at high velocity soft-block within a few hundred requests; residential proxies, pacing and session rotation push that far higher.
Profile vs hashtag vs post — which is hardest?
Hashtag is the most gated and fragile, profile is the most reliable, and single-post reads sit in the middle (the comment thread is the gated part).
Disclosure: links to Apify on this page are affiliate links. If you create a paid account through them we may earn a commission, at no extra cost to you. We recommend Apify because we build and ship Actors on it ourselves.