Project Goals¶

The motivation for this project is to download, process and analyse a data set of Steam apps (games) from the Steam store, and gain insights into what makes a game more successful in terms of sales, play-time and ratings. We will imagine that we have been approached by a company hoping to develop and release a new title, using the findings we provide them to inform decisions about how best to manage their budget and hopefully increase the success of their next release.

The first step will be tackling data collection – the actual retrieval of data from Steam’s servers and databases. In the future we’ll look at cleaning the data, transforming it into a more useful state, then on to data exploration and analysis. Finally we’ll summarise our findings in a non-technical report which would be sent to the fictional company in question.

At the end of the data collection and cleaning stages, we’d like to end up with a table or database like this:

name id information owners price rating awesome game 100 genres, descriptors, variables 100,000 £9.99 9/10 generic shooter 4 200 definitely the best shooter 50,000 £39.99 6/10 … … … … … …

We can then interrogate the data, and investigate whether particular attributes tend to result in more successful games. Metrics like ownership and ratings should help define the success of a title.

Data Acquisition¶

There are a number of ways to get this information. Obviously we could search the web (and especially kaggle) for existing datasets, however to avoid letting someone else get away with all that hard work (and mainly for the purposes of learning) we’ll be acquiring all the data ourselves from scratch.

Often when generating data the best place to start is to check for APIs. Fortunately Valve (the company behind Steam) make one available at https://partner.steamgames.com/. An API such as this allows anyone to interface with data on a website in a controlled way, usually providing a host of useful features to the end-user. Typically an API is a great way for developers to allow access to databases and information on a server. Unfortunately this documentation doesn’t include all access points, but others have documented this for us. This documentation of the StorefrontAPI will be particularly useful.

We’ll be able to get good information about the details of each game from the Steam API, however we’re still missing information about popularity and sales. Luckily we can easily get this data from another website, SteamSpy.

SteamSpy is a Steam stats-gathering service and crucially has data easily available through its own API (documentation here). It provides a number of useful metrics including an estimation for total owners of each game.

We’ll be retrieving data from both APIs and combining them to form our dataset. For the purposes of this project, we’ll be performing as little data cleaning as possible at this stage, providing ‘dirty’ data for data cleaning, the next step in this project.

Section outline:¶

• Create an app list from SteamSpy API using ‘all’ request
• Retrieve individual app data from Steam API, by iterating through app list
• Retrieve individual app data from SteamSpy API, by iterating through app list
• Export app list, Steam data and SteamSpy data to csv files

API references:¶

• https://partner.steamgames.com/doc/webapi
• https://wiki.teamfortress.com/wiki/User:RJackson/StorefrontAPI
• https://steamapi.xpaw.me/#
• https://steamspy.com/api.php

Import Libraries¶

We begin by importing the libraries we will be using. We start with standard library imports, or those available by default in Python, then import the third-party packages. We’ll be using requests to handle interacting with the APIs, then the popular pandas and numpy libraries for handling the downloaded data.

Top 4 steam api doesn't return the url to download the file edited by Top Chef