Gateways

Gateways are responsible for indexing, serving, and making data available to users.

Indexing

To ensure that all transactions on Arweave are easily searchable and accessible, gateways index all transaction metadata, including transactions posted via Irys, and transactions posted directly to Arweave. Gateways monitor Arweave for new blocks, and indexes them as they are propagated. Whenever a user posts a transaction to Irys, gateways detect it and add it to its own database.

During the indexing process, gateways extract transactions from each block and index the metadata. When a bundle is encountered, gateways index all the transactions within. Similarly, if a manifest is found, gateways indexes it. By indexing manifests, gateways create a virtual directory structure that mimics the original folder system. This allows users to access data using the URL format https://gateway.irys.xyz/[manifest-id]/[file-name].

Querying

Transaction metadata can be queried directly using the Irys query package a REST API or GraphQL. This enables users to create tailored queries, such as searching for specific tags or addresses.

Irys supports adding custom metadata tags to each upload, these tags allow users to build data models, all queryable via GraphQL.

When a user posts a transaction to Irys, gateways pick it up and store it in their database. This allows gateways to support optimistic GraphQL, meaning query results include both transactions seeded on Arweave as well as those expected to be seeded soon. This ensures users have access to the most up-to-date data and can use it to build real-time applications.

Serving Data

When a transaction is seeded on Arweave, its header and data are stored separately (as explained in transaction lifecycle). To access this data, users request it from a gateway. Because it can be complex, computationally heavy, and difficult for average users to access data directly from miners on Arweave, gateways are crucial in making downloading data as simple as uploading.

When a user requests data from a given gateway, the gateway first checks its internal cache, serving the data from there if it's available. If not, the gateway then checks Arweave to see if the data is stored there. If found, that gateway retrieves the data and serves it to the user, caching it for future use and providing faster response times for frequently requested data.

If the data is not found on Arweave or the gateway’s local cache, the gateway checks Irys' optimistic cache to see if the data is available. By doing this, the gateway allows for instant access to data uploaded to Irys, even if it hasn't been finalized on Arweave. This also means data uploaded to Irys' Devnet, where uploads are kept for approximately 60 days, is available via gateways.

Scaling

Gateways are a crucial component in Irys' scaling strategy. To effectively scale, it's important to separate read and write components so they can be optimized independently. In Irys' approach, bundlers handle writing while gateways handle reading.

Gateway schema / endpoints

To download data from a gateway, use the transaction id obtained during upload. The most commonly used gateway is https://gateway.irys.xyz, and the format used to download data is https://gateway.irys.xyz/[transaction-id].

For example https://gateway.irys.xyz/CO9EpX0lekJEfXUOeXncUmMuG8eEp5WJHXl9U9yZUYA.

For GraphQL queries, you can connect to either of these endpoints: