Road to 1.0

[coder4life]

Introduction

Welcome to TargetDummy, testing application for Battle.net APIs. There are several primary and secondary goals for this project overall. The ultimate goal is to have a consistent set of testing methodology that can apply to all API endpoints.

Summary

Due to the various inconsistencies, errors, availability, and accuracy of the output of various endpoints, it was determined a more desired testing methodology was required to capture and report errors. Whenever working with a high mutating data source, such as game database engines, there is a high probability that something will break between big releases. Many times these issues have been documented but lost to time, inaccurately reported, and/or accepted as is.

Going forward the project is attempting to address two fronts of perspectives:

1. The consumers of the API and how to effectively report issue and create test cases on those issues
2. The developers of the API (Blizzard) and how to effectively continuously report ongoing and new issues.

The checklist below will outline the progress of completion of the various tasks. Anyone is welcome to pick up an area of the project and help with development.

Checklist

Note: These will not be completed in any particular order, a small description will be given next to each item to state the intention behind the decisions of each process.

Pick: Languages / Server Technologies / Database Technologies

A basis for the technology to be used. The decisions here will guide the direction of how the various technologies will be used. The cost and benefit analysis will determine the "best fit" model for the intention of the project going forward.

• Runtime language for processing API related tasks (requests, logging, testing, reporting) – JavaScript async run-time environment using NodeJS. NodeJS, although the new kid on the block, provides an industry movement on async event operation. At its core NodeJS is single-threaded, however its event based delegation is a great match for running a lot of concurrent tasks that do not necessarily depend on state based operations. Although the majority of our developers are PHP based and it has been proven that a similar result could be achieved through a convention deployed by React PHP, it was determined the consequences would outweigh the benefits for the type of requirements that must be deployed.
• Public facing API information for the general public (documentation, examples) – LAMP stack with Grav CMS modified for a custom template based on Stripe Docs. Grav is a fast PHP CMS that is meant for small PHP applications. At its core it supports markdown and folder structuring as such all docs can be stored and viewed on a version control system like git. The reason against a prebuilt cloud based document system such as Apiary was determined to have some substantial caveats of "entice to pay". With the knowledge that general tools could provide for free and better integrate with our workflow.

Interface

The Interface is a way to collect and analyze data. Currently discussions on the API and the associated reports are processed over various log formats. It is hard to get a consistent scope regarding issues in the API (1) who the issue effects (region, certain accounts, ect.), (2) what the issue is (known issue, up-time issue, error, ect.) (3) where exactly the issue occurs in the request response (response header, response body, ect) (4) why the issue may arise (downtime, new patch, game DB changes, ect.) (5) how the issue may be fixed.

BNetTargetDummy/Application

https://github.com/BNetTargetDummy/application

• DiscordBot – The ability to run various console commands from discord itself to provide requests, log requests, run tests, and report tests. Some commands will be permission gated.
• • Help – Output help prompt for all commands.
• • Log – Log an endpoint resource. Return log result based on endpoint response, and output results to the current channel Note useful when logging an endpoint behavior.
• • Ping – Ping the bot to see if bot is alive.
• • Report – Report based on the responses of previous endpoints.
• • Request – Request an endpoint resource. Return a link to the endpoint response to the current channel. Useful when discussing about various fields of data in the response. Note results are cached but not permanently stored.
• • Test – Test using predetermined cases on an endpoint resource and output results to the current the current channel. Note useful for determining the status of known test cases.
• • Version – Ping the bot to see if bot is alive.
• Express - Express server will be used for basic REST interface for basic data. We are planning a more full featured application.

BNetTargetDummy/console

https://github.com/BNetTargetDummy/console

• Commands – CLI that can run standalone as part of a project. It was decided to take advantage of @benweier Blizzard.js library. It is important to note BlizzardJS is missing some endpoints still due to the new Application OAuth2 flow needed.
• • td request scope resource
• • • td request profile – Community OAuth Profile APIs endpoints [eu, kr, sea, tw, us]
• • • • td request profile account
• • • • td request profile profile
• • • td request data – Game Data APIs endpoints [eu, kr, tw, us]
• • • • td request data d3
• • • • • td request data d3 season
• • • • • td request data d3 era
• • • td request d3 – Diablo Community APIs endpoints [eu, kr, tw, us]
• • • • td request d3 data
• • • • td request d3 era
• • • • td request d3 profile
• • • • td request d3 season
• • • td request sc2 – Starcraft 2 Community APIs endpoints [eu, kr, sea, tw, us]
• • • • td request sc2 data
• • • • td request sc2 ladder
• • • • td request sc2 profile
• • • td request wow – World of Warcraft Community APis endpoints [eu, kr, tw, us]
• • • • td request wow achievement
• • • • td request wow auction
• • • • td request wow boss
• • • • td request wow challenge
• • • • td request wow character
• • • • td request wow data
• • • • td request wow guild
• • • • td request wow item
• • • • td request wow mount
• • • • td request wow pet
• • • • td request wow pvp
• • • • td request wow quest
• • • • td request wow realms
• • • • td request wow recipe
• • • • td request wow spell
• • • • td request wow zone
• • td log request_id
• • td test group_name
• • • all - Runs all tests, tests my have subgroups of items.
• • • access – Tests just to check state of all endpoints
• • • schema – Check keys and the associated value types
• • • compare – Compares multiple tests of the same endpoints but different data
• • • user – Specific item testing created by consumers of the API, like checking of an endpoint is returning the correct data in the fields
• • td report output_interface
• • • td report email – Able to send a report to a specific email using a formatted email. We would send information (specifically our group) to Blizzard API Team every few days using the api-support@blizzard.com. Thinking of a status email with a list of test reports (green, yellow, red) in email with basic status text. A link would take you to an analysis page to find out why the test failed. A comparison between last test weeks to see if any were fixed. Better feedback and tracking for us and you.

BNetTargetDummy/logger

https://github.com/BNetTargetDummy/logger

• Store data
• Support SQLite, MySQL, flat-file.

BNetTargetDummy/site

https://github.com/BNetTargetDummy/site

• Site template – A multi-theme capable template for GetGrav CMS for Introduction Pages, Documentation, API Documentation, Blogs, and general information. Using Bootstrap, Particle-JS, OwlCarousel2, jQuery
• Site structure – Site will have many top level pages and sub-pages.
• • Home – Advertising page for TargetDummy
• • Docs – Docs – Contains docs for the various parts of TargetDummy
• • • Application Intro
• • • Console Intro
• • • Logger
• • • Test
• • API – Contains clearer docs on Blizzard APIs (parameter types and values, example code)
• • • Introduction
• • • Blizzard Mashery user account and application process
• • • OAuth
• • • • Application OAuth Flow
• • • • Profile OAuth Flow
• • • • OAuth libraries per language
• • • Conclusion
• • Blog – Whatever we want to talk about: development challenges, Blizzard API news, Blizzard employee interviews, guest writing, ect.
• • About Us – Acknowledgement page of sorts.

BNetTargetDummy/test

** Not started yet ***

• Test framework – Will hosts the testing app code to be able to run tests. Tests will be stored with the console app however. Mocha for general test framework MochaJS and Chai for assertion ChaiJS.
• • Access - Does the API respond at all and with an expected response (200, 404, ect.).
• • Schema – API returns the expected value types (boolean, number, string, key, ect.). Example can be seen here Auction API is Null. Swagger spec, exclusively used for schema validations, in favor of Apiary based documents. Although Apiary provides the benefit of being markdown with some "sugar" syntax inline, its flat structure and unorganized nature may be difficult to traverse. Plan is to be able to generate these formats on the fly eventually.
• • Compare – Compares multiple tests of the same endpoints but different parameters and options. The goal here is the keys are expected to be the same, and certain values are expected to be the same but do not necessarily have to be as it depends on the expected result specification.
• • User – User driven tests, involves user driven tests developed in JS to test for appropriate conditions (parameters and options) with expected values (range, quantifier, ect.). Example given can be seen here Leaderboards Not Returning Expected Values.

Conclusion

We are having weekly on day on the weekend 15-30 minute meetings to better keep track of challenges, design decisions, and goal tweaks. We have small burst chats on the discord channel group when we need to clarify a few things.