Skip to content

Evaluation: Introduction

This section outlines the procedure for participating in the AH2AC2 challenge evaluation.

Evaluation Steps

The evaluation process involves the following key steps:

  1. Pre-registration: To participate, teams must first pre-register their planned experiment via our submission system, available at ah2ac2.com. This step helps us understand your approach and plan accordingly.

  2. Submission Review: After you submit your pre-registration, our team will review it. We will check for completeness and adherence to the challenge guidelines.

  3. API Key Provision: Once your submission is approved, we will provide you with two distinct API keys:

    • Test API Key: This key grants unlimited access to a test environment. You can use it extensively to test your agent's integration with our evaluation API and ensure everything works smoothly. In this test environment, your agent will interact with opponents making random moves, but the API interaction mechanics will be identical to the official evaluation.
    • Evaluation API Key: This key is for the official evaluation and has a limited number of uses. Please use it carefully and only when you are confident your agent is ready for the official runs.

  4. Results: After all evaluation games are processed for your submission using the Evaluation API key, your results will be made available on the official challenge leaderboard, which can also be found at ah2ac2.com.

Getting Started with the API

It's important to note that the dataset for the challenge is available in advance. You do not need to wait for submission approval to access the dataset or to begin integrating with our API using the provided documentation.

This documentation, particularly the Evaluation Usage Example, focuses on Step 3: guiding you through the process of using your API keys to implement your agent and successfully integrate with our evaluation system.