API project tech discussion (share your details)

Other Halo API Support
1

lazerhawk05 suggested I start a thread in regards to the community API project technical stuff for discussion. I hope others are willing to share some of their experience as well (tech used, problems overcome and such stuff).

Halo 5 Statistics for Microsoft Band
About me: 30 yr old male (new dad) in southern Ohio that has worked in government contracting, education, and healthcare industries as a software developer. BA in Computer Science.

Project Tech Used:
Hosting: RedHat OpenShift - www.openshift.com (although I was originally looking to use Azure, I couldn’t find a pricing tier that worked well for me) redhat was is free :slight_smile:
Server Side: Java webapp using Spring
Client Side: HTML5, Bootstrap (responsive design), jQuery, Font-Awesome
Tracking: Google Analytics via client & server side (this part necessary because I have to capture traffic that isn’t from a website but from a band/app)
Specific difficulty: The most difficult part of my application was actually the emblem & weapon image manipulation. I had to bring in the images in through the API and flatten out to one color + transparency then resize it down to very small sizes without loosing too much quality. Followed by packaging it all up into the file that is delivered to install on the Microsoft Band. That doesn’t sound to difficult but reducing images to 46x46 or even 20x20 can make things pretty unrecognizable. To this day, some emblems in the game show a lot better than others because of the nature of the image. Those that are solid, just look like blobs while others look amazing.

Quick story:
I’ve owned my Microsoft Band 2 for a about a year now and I have been a bit disappointed by the lack of 3rd party developer support for the thing considering the great documentation out there and the cross platform support on iOS, Android, and Windows Phone. It makes me sad that such a great product has flown under most people’s radar. I would even say that Microsoft has missed an opportunity by not having their first party games somehow tie into this smart-watch. I wanted to make something for it but never had the motivation to start. In the past I’ve published two games on Xbox 360’s XNA Indie Games Marketplace aside from that, all development has been for school or work.

These days, I generally don’t program at home. After a long day writing code at work, usually it is more fun to kick back and blow off some steam in Halo or Destiny. However, now with the reduced free time in my life, I’ve come to value the little I have. Spending all of that on gaming (as much as I love it) seems like it might be wasting some. If I’m going to lose sleep or time with family & friends, it better be for a purpose.

However, I was excited at the opportunity to enter this competition. I figured, this was my best chance in a contest since there probably aren’t TOO many developers out there ready to put the huge effort required (wow was I surprised by the quality of projects!). I’ve been telling myself I’d get back to programming in my free-time and this was the result.

Passion drove this project. I knew from the beginning most people don’t have a Microsoft Band to even try this out but I hope the community appreciates that it exists. After many many late nights (a few realizing at 5am that I had to get up for work in an hour or two). I kept adding functionality every day until the contest was over. At this point I can say that hundreds of hours have gone into this(worth every second).

I posted on gaming forums and heard back from many people loving what I had built. Some people asked for specific functionality while others for support for more games. Since the contest conclusion, I’ve been able to rapidly fix bugs, and add functionality. Today, I’m proud to support both Halo and Destiny with Battlefield, Diablo, and more in the pipeline. I’ve enjoyed my little niche - developing for something not many other people have. I wanted my project to be different and I think I accomplished that.

Now that the contest is over, I’ve reverted to my old ways casually improving the app in my free time while also enjoying the Warzone Firefight release and playing Raids in Destiny.

Also, I wanted to give a shout out to my amazing wife who thought this wasn’t a waste of time and took up more than her fair share with the baby while I had my head deep in code. Amazing she was encouraging me to enter a contest about a game while she is changing diapers/feeding/etc. Love you!

Thanks for taking the time to read!

If we share a little about ourselves as well as the technologies we’ve used, that might help some fledgling developers get a start on their project, education, and possibly career.

Edit: Added quick story

2

This is a nice Idea. I might have written mine a bit too personal but it covers the technology I used and some other insights into my experience.

My submission: HaloTheater.com
About me: I’m a 27 year old guy from England with a degree in Computer Science. I’m currently developing a mobile game while doing the odd bit of freelance work. I’m hoping to get a job in the games industry soon in either QA or a developer role. Halo is my favourite game franchise and I’ve been playing it since CE.

I think it’s worth mentioning:

  • I had almost no web development experience prior to this project. - I was however able to work on it pretty much full time for most of the hackathon. - I decided to enter the hackathon because it would be a great excuse to learn a new set of skills and something I could add to my CV.Server Side:
    I’m currently using Visual Studio and C# a lot so it made sense to go with Microsoft’s ASP.NET MVC web framework. This was surprising easy to get up and running. It also meant I could us Furiousn00b’s brilliant C# wrapper for accessing the API and rate limiting. Thanks again! All the data I collect from the server is stored in a database and cached in memory on the server.

Client Side:
HTML5, Javascript, JQuery, D3.js
I used D3.js to organise all of the match/event data and handle all of the drawing/playing of the Match Viewer. This library is absolute fantastic; it’s very powerful for data visualisation and was easy to learn.

Hosting:
It’s currently hosted on a Windows machine on Amazon’s AWS. I’ll probably look for something more affordable soon as it’s a little pricey!

Project Timeline:
I had a few potential projects in mind and fortunately I’d started playing around with the API a week or so before the hackathon was announced. I was initially planning to collect as much data as possible from the API so I could run interesting queries such as:

  • Playlist popularity - Global weapon/medal stats (rarest medals etc) - Personal stats like “How many times has [GamertagA] killed [GamertagB]” - Live playlist population estimations - Leaderboards etc.I spent a lot of time defining my own database schema to store every single bit of data that can be returned from the API. I ended up with about 40 tables and hundreds of fields. I then spent far too much time writing my own ORM which was a bit silly, but enjoyable. I pulled the entire match history (stats, not match-events) for 4 or 5 players and the database was close to 1GB in size… At this point I decided my initial plan might not be viable! But the database would still be useful.

I decided to move onto what would become HaloTheater. I did have the ludicrous idea of flying cameras around every inch of a map and performing 3D reconstruction from the data so the maps/events could be viewed in 3D. I was doubtful it would be feasible but had a quick look into it anyway; it wasn’t. I then thought that photo-spheres (Like google street view) would be a cool alternative; this might have been doable but I ended up not pursuing it.

So in the end I decided I’d do something inspired by Bungie’s old website. Although I’d do my best to improve on it by making it interactive/filterable. Unfortunately I didn’t know any of the camera mathematics required, but how hard could it be!?

Two or three days later and after watching tens of hours of lectures online I had a working camera class that could project points from the 3D game-event coordinates to screenspace for any camera I defined. This was possible using the machinima controls available in Halo 5 which allow you to display the camera properties in game. I then thought it would be nice to take the screenshots in stereo pairs and try to compute a depth map; this would allow me to occlude events that were behind walls. Again this was a little too ambitious/impracticable so I didn’t go with it.

Once I knew my project was possible (this time) I started to build the site. You’ll notice my site is very… bland. I had little/no experience with HTML, CSS and Javascript and spent an embarrassing amount of time trying to my divs/css to behave! For this reason I tried to keep the layout as simple as possible and focus on the functionality of the event viewer.

A few days before the deadline I was fairly happy with what I’d implemented although many ideas had to be thrown out/postponed. All that was left was to take the screenshots and add the cameras to my database… once again I drastically underestimated how long this would take me!

I was already pretty burnt out at this point but, with some setbacks, it took me almost 3 days solid to take all of the screenshots and record all of the camera properties. It was painful. In the end I managed the 12 arena maps with up to 32 cameras on some of them. I was far too particular when positioning the cameras and for each camera I had to take 2 screenshots, one with camera coordinates displayed and one without. I then renamed every screenshot by hand, numbering them in the order I wanted them to appear and naming each one based on what it was looking at in game! I used FFMPEG to bulk convert all of the screenshots to 720p jpegs.

If anyone wants to implement something similar here are some useful things I found:

  • Use the machinima controls in a custom game to fly around and see the camera properties - Use Forge to remove weapons and power-ups from the maps before taking screenshot as they will likely be moved in the future. - Forge also allows you to add a “Filter” that increases the game’s field of view, this gives each screenshot slightly increased map coverage. - You can get outside of almost any map by placing your Spartan’s back against an outer wall of the map and then switching to the free camera view (machinima controls). The camera spawns behind your spartan and therefore outside the map. In this example you can see many of the cameras are outside the map: http://halotheater.com?matchId=9abba264-5732-4e9c-9539-e1f99ab607f4All in all it was a pretty large amount of work, but a really good learning experience and very enjoyable (mostly)! I’m pleased with how it turned out even if some of it was pretty rushed! After this experience I’ll definitely be looking at creating more websites in the future. I’m particularly interested in working with future Halo APIs and API’s for other games.

Cheers.

3

Fantasy Halo 5 (fantasyhalo5.com)
About me: 28 year old male (father of 3 - 3 years and under) in Oregon. I have a BS in Elementary Education … so a little different then seph and super shredder. I started working in tech 6 years ago troubleshooting educational technology issues in classrooms. When I began this project the only experience in coding was a couple python tutorials that taught me how to define def’s and print. So … this was a huge learning curve for me. I currently am a systems engineer so I know a lot about the server side and how to get my server the way I want, but the app side was all new. I started my project after playing with the API for a day or two to pull in some friends stats about 3 weeks before it was due. Little did I know all the languages I had to learn

Server Side:
I settled on an Ubuntu server because that is what I am most familiar with in my job. I also created a mysql database that is now hosted on Amazon RDS. The connection was easy and the server side to allow apache to use Python was pretty simple for me as I do that almost daily at my job. This was the only area I actually felt like I knew what I was doing.

Hosting:
I chose AWS to host my Ubuntu server, because I was able to do a year free with a Ubuntu server and an RDS instance, which is my SQL database.

Client Side:

  • HTML5
  • CSS
  • Flask – A Python interpreter to allow me to have variables and templates in my HTML5 code. (pretty awesome add on and was fun to learn)
    I didn’t have time to learn javascript, which is a bit disappointing, because I know there is great potential there, but I am working on that.

Tracking:
I track users in Google Analytics. It was pretty fun to see the traffic role in even though it has died out a little bit.

App Story
This app started out with a couple of my friends who ended up working with another developer on fantasyhcs (if you haven’t checked it out you should it is pretty cool) telling me they wanted a fantasy league for Halo. I told them I would look around, but not sure if I could complete it before this seasons HCS. So I started playing with the API in python and pulling in our stats.

Then they extended the API competition time … I noticed that on May 8th ish. So I decided lets learn. I spent almost every night after that, once I got my 3 kids down for bed and hung out with my wife, down in my basement coding. I only slept about 2-3 hours per night at that point. I had to learn Python, Flask, MySQL, HTML5, CSS and Jinja2 templates. I had minimal knowledge in some of those, but not enough to do most of what I needed to get done. So I did a lot of google searches.

Major stopping points that took a while for me was getting my pages to load fast enough that you wouldn’t walk away and still have the custom information that I wanted. I also had to create a user base structure on how to encrypt passwords (so I don’t see them and never could) and make sure that usernames where only used once. Pretty standard stuff, but for me really hard. I ended up using a hash to encrypt the password store it and ran scripts that check the database for existing usernames to ensure there is only one. Had a few kinks in this so I spent multiple hours working through the scripts to make sure I really couldn’t create a second user and the password was unseen.

The hardest part for me and if you go to my site you will notice. If you have are not a recorded player in my site and you get looked up for stats then you will not have any if it is the first time you visit until after two minutes. I still can’t get the info form Halo and into my database for you user stats with a recorded time in under 2-3 seconds. I got it down to 2 seconds, but then I tried to put it in my database and it failed constantly. So that is an area I am still trying to figure out how to do. Page loading speed to me was very important and now that my slayer_stat table has over 75k rows it makes it a little harder :slight_smile:

All in all I had a blast learning how to code and plan to continue my work in Python as I move forward since then I created the dismas his own little reporting tool that gets data and puts it out in a csv (GitHub - jphilipsen05/halo5quickstats: Currently set up to pull in the Halo API's get_arena_service_record and get_warzone_service_record. More API's will be added as time goes.). I also have already purchased the domain fantasyhalo6.com which will be fun to build and probably be much more functional when I know more about Javascript and the other applications.

I thank 343 industries for setting this up and basically giving me a reason to quickly learn how to code. I can’t wait for the winners to be announced … Even though I’m pretty certain I’m not winning after I looked at all the other entires :slight_smile:

Thanks super shredder for setting this up and getting this conversation going. It has been fun to read the few posts. The projects that came in are pretty amazing … I’m still holding out for an in-game developer emblem :slight_smile:

4

Halo Carnage (www.halocarnage.com)
About me: I’m a 21 year old from Australia who started this project when I finished my Bachelor of Information Technology degree. I’m currently working in a full time, non-IT related role while looking for a better suitable position. I’ve only learned the basics of programming and databases, however I have had some experience in previous programming roles. I’ve loved Halo as a kid with Halo CE, and have been currently competing in some ESL Aus games!

Hosting: It’s my first experience with cPanel for PHP and MySQL.

Server Side: I borrowed a spot on my brother’s rented server in Australia (which made load times difficult for most users) and had an 8GB database to play with.
Client Side: HTML5, JavaScript, Google Visualisations. I built the iOS app on Swift, the Android app on Java/Eclipse and the Windows app on .NET (C# I think)

Tracking: Google Analytics for user and device tracking <3

Specific difficulties: The maps and the API loading times. For the maps I needed to use machinema mode in game to track player coordinates, as I walked around the level to map out the walls/ramps/stairs in Photoshop on a 10:1 ratio for pixels and in-game coordinates. This was extremely difficult because I had never used Photoshop before. As I had also never used PHP before, the next difficulty was loading times for processing and for getting API responses. The server is based in Aus and I didn’t have a full understanding of the API while I built the structure of the website. This means the browse section had quite a delay as I attempted to cache games so it was quicker to view the games individually. This was the same for graphs, which users commented that the graphs impacted their performance.

Making the site mobile compatible was also a struggle!

Quick Story: This was my first IOS, Android and Windows app. All technologies were new to me, I needed to borrow a Mac from a friend to work on the iOS app. And it was my first time developing with PHP, and the first time making a mobile compatible app. This was all extremely difficult for me, but it’s been the funnest project I’ve been on, and I didn’t think I’d take it this far but I’m so happy I’ve spent so long working on this!