# Running Locally

## Running the Transaction Stream Service Locally <a href="#running-the-transaction-stream-service-locally" id="running-the-transaction-stream-service-locally"></a>

{% hint style="info" %}
This has been tested on macOS 13 on ARM and Debian 11 on x86\_64.
{% endhint %}

When building a custom processor, you might find it helpful to develop against a local development stack. The Transaction Stream Service is a complicated, multi-component system. To assist with local development, we offer a Python script that wraps a Docker compose file to set up the entire system.

This script sets up the following:

* Single node testnet with the indexer GRPC stream enabled.
* A Redis instance.
* Transaction Stream Service, including the following components:
  * [cache-worker](/endless/devbuild/build/endless-apis/grpc-transaction-stream/running-locally.md): Pulls transactions from the node and stores them in Redis.
  * [file-store](/endless/devbuild/build/endless-apis/grpc-transaction-stream/running-locally.md): Fetches transactions from Redis and stores them in a filesystem.
  * [data-service](https://github.com/endless-labs/endless/tree/main/ecosystem/indexer-grpc/indexer-grpc-data-service): Serves transactions via a GRPC stream to downstream clients. It pulls from either the cache or the file store depending on the age of the transaction.
* Shared volumes and networking to hook it all up.

You can learn more about the Transaction Stream Service architecture here and the Docker compose file [here](/endless/devbuild/build/endless-apis/grpc-transaction-stream/running-locally.md).

### Prerequisites <a href="#prerequisites" id="prerequisites"></a>

In order to use the local development script you must have the following installed:

* Python 3.8+: [Installation Guide](https://docs.python-guide.org/starting/installation/#python-3-installation-guides).
* Poetry: [Installation Guide](https://python-poetry.org/docs/#installation).
* Docker: [Installation Guide](https://docs.docker.com/get-docker/).
* Docker Compose v2: This should be installed by default with modern Docker installations, verify with this command:

```
docker-compose version --short
```

* grpcurl: [Installation Guide](https://github.com/fullstorydev/grpcurl#installation)
* OpenSSL

### Preparation <a href="#preparation" id="preparation"></a>

Clone the endless-core repo:

```
# HTTPS
git clone https://github.com/endless-labs/endless.git

# SSH
git clone https://github.com/endless-labs/endless.git
```

Navigate to the `testsuite` directory:

```
cd endless-core
cd testsuite
```

Install the Python dependencies:

```
poetry install
```

### Running the script <a href="#running-the-script" id="running-the-script"></a>

**Starting the service**

```
poetry run python indexer_grpc_local.py start
```

You will know this succeeded if the command exits, and you see the following:

```
Attempting to stream from indexer grpc for 10s
Stream finished successfully
```

**Stopping the service**

```
poetry run python indexer_grpc_local.py stop
```

**Wiping the data**

When you start, stop, and start the service again, it will re-use the same localnet data. If you wish to wipe the locnet and start from scratch you can run the following command:

```
poetry run python indexer_grpc_local.py wipe
```

## Using the local service <a href="#using-the-local-service" id="using-the-local-service"></a>

You can connect to the local Transaction Stream Service, e.g. from a custom processor, using the following configuration values:

```
indexer_grpc_data_service_address: 127.0.0.1:50052
auth_token: dummy_token
```

You can connect to the node at the following address:

```
http://127.0.0.1:8080/v1
```

## Debugging <a href="#debugging" id="debugging"></a>

**Usage on ARM systems**

If you have a machine with an ARM processor, e.g. an M1/M2 Mac, the script should detect that and set the appropriate environment variables to ensure that the correct images will be used. If you have issues with this, try setting the following environment variable:

```
export DOCKER_DEFAULT_PLATFORM=linux/amd64
```

Additionally, make sure the following settings are correct in Docker Desktop:

* Enabled: Preferences > General > Use Virtualization framework
* Enabled: Preferences > General > Use Docker Compose V2
* Disabled: Features in development -> Use Rosetta for x86/amd64 emulation on Apple Silicon

This script has not been tested on Linux ARM systems.

**Redis fails to start**

Try setting the following environment variable before running the script:

```
export REDIS_IMAGE_REPO=arm64v8/redis
```

**Cache worker is crash-looping or `Redis latest version update failed.` in log**

Wipe the data:

```
poetry run python indexer_grpc_local.py wipe
```

This means historical data will be lost.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.endless.link/endless/devbuild/build/endless-apis/grpc-transaction-stream/running-locally.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.