Medicine and the world of gaming do not often mix well in the public eye. Whether it be the old debate of violence in video games or the stigma around our hobby, there has been quite a bit of pushback in recent years. However, with organizations such as Checkpoint popularizing studies into the minds of gamers, medicine and video gaming are moving closer together one day at a time. One company, Balanced Media Technology, is looking to harmonize that connection further, and have spent the last several years developing a medical AI that learns from the behaviors of gamers as they play their favorite games. We sat down with BMT’s Dr. Corey Clark to ask him about the company’s developments.
Currently BMT is working on developing HEWMEN, its crowd-sourced platform to collectively train Artificial Intelligence via assistance from outside human guidance. How is player input shaping the development of this program?
HEWMEN not only can help train Machine Learning algorithms, but it allows players to be part of the computations in the algorithms as well. We can combine player inputs to create a collective result for a given problem. In one of our current applications the results are not the actual answer to the problem, but an averaged intuition of where players believe an answer may be located. This provides a filter for the machine learning algorithm that helps guide and focus where to use its processing. In this case, humans are not “training” the ML algorithm, but rather working collaboratively with it and helping provide an intuition to guide the ML technique to specific areas, or to give further insight on the input data.
BALANCED follows standard techniques when it comes to user experience in games, building and testing iteratively to help refine the core game and engagement loops. Having fun and keeping players engaged is paramount, and is always the core focus. Once we know players are having fun and are engaged, then it’s time to refine the human computation task that are happening under the surface.
BMT’s website mentions that HEWMEN, “combines human perception… with raw computational power by utilizing volunteer grid computing.” How is HEWMEN collecting usable data as Gamers play, and how do you go about verifying that data afterwards?
Data sets, such as optical coherence tomography (OCT) scans, are provided by our partners. These data sets are used to help define levels and interactions in the game. Once a player begins to interact with the data through the patent pending HEWMEN game interface, it provides either a refinement or the generation of completely new data set. The modified data sets are then stored locally on players’ machines or sent back so results can be analyzed. We also utilize multiple verification and validation techniques on data that is reported back. A problem is not being solved on a single machine in isolation, it is actually replicated and repeated across multiple games/machines all at the same time. This allows for results to be compared and combined to remove any results that do not fit with the collective.
With every game there is a failure state where most gamers will achieve a, “Game Over,” as they play. As the focus of HEWMEN is the application and development of medical technologies, how does an achieved failure state impact HEWMEN’s development and data?
While games built using HEWMEN are able to process data and provide human insight, their game design does not have to be focused directly on the problems they are solving. HEWMEN provides an abstraction between data/problem and the game. This allows the game to have its traditional gameplay loops, where HEWMEN tasks are supplementary. Therefore, a failure state in the game doesn’t directly relate to a “failure” in data/problem processing.
Currently HEWMEN has been integrated into your company’s original games, 8 Cell and Eye in the Sky, both of which are incredibly niche and focused. How can you see HEWMEN being more widely integrated into broader and more popular games?
We have already integrated HEWMEN into existing games such as Minecraft and Q*bert. HEWMEN has integrations into Unity and the Unreal Engine that allow game developers to integrate with our technology and provides an interface for data/problem processing. These integrations allow for developers to work with HEWMEN directly in the tool sets and pipelines they are already familiar with, as well as provide an abstraction to the actual algorithms running “under the hood”. Our goal is to show multiple common gameplay loops found in existing games to use as examples for developers. Once they see how common gameplay mechanics are being used and integrated with the HEWMEN interface, it opens to the door to integrate into existing games as well as create all new games.
Previously BMT has worked on several mods for the popular PC game Minecraft to develop co-medications for several diseases. How does this carry back to the development of BMT’s in-house games and HEWMEN?
We use everything we learned from working in Minecraft as well as our partnerships with Feed The Beast (FTB) and CubeCraft to help further the development of HEWMEN. Having an opportunity to work with Minecraft and our partners helps prioritize features in our own infrastructure and development tools. The experience is also used to think through new game mechanics that could be used as well as new types of problems and data sets that could be utilized within the same game environment. In fact, the Minecraft mod used in the alpha event was not limited to only the co-medication chemotherapeutic problem, it is capable of working on multiple problems at the same time. This is one of the great features of HEWMEN — games are built around data and techniques, not problems. This allows the same game to be used over and over but not limiting it to a single problem. Working with Minecraft has allowed us to test, deploy and validate our approaches and techniques.
Are there any plans currently for a wider release of HEWMEN integrated games on platforms such as Steam or Green-Man Gaming?
We have plans to release HEWMEN across all gaming platforms. The HEWMEN feature sets available to games or applications may have limitations depending on the platform, but all should be able to doing some sort of human computation task at a minimum. Our immediate goal is to focus on building demos and integrations for Unity and the Unreal Engine. We have already been approached by a number of developers interested in integrating HEWMEN into their existing and/or upcoming games and having these demos and integrations should help support these efforts.
With the human input and active engagement through gaming, what is the end goal for the development and release of HEWMEN? While this iteration is clearly being tooled to work in the medical field, what is BMT’s ideal application goals for HEWMEN?
BALANCED’s current focus is to bring the combination of human computation and machine learning to the medical field, but HEWMEN is by no means limited to this industry. Our goal is to show its effectiveness in the medical space and from there allow third parties to create HEWMEN enabled applications and games that expand into other markets. We have already been contacted by companies in other industries such as transportation, satellite imagery and cyber security to look at opportunities in their specific use cases. HEWMEN is a platform, and our ultimate goal is to open it up to developers so they can build out a new breed of software applications, one where humans and machines work together in the same environment, collaborating while doing things never before possible.
With the current media attitude towards gaming and studies thereto, have you found any difficulty or stigma promoting BMT and HEWMEN in the medical fields?
BALANCED’s HEWMEN platform combines multiple areas of expertise and has the ability to impact any industry it is applied to. This large-scale ability is a challenge to explain in a 5min chat, and understanding the technology is one of the most challenging obstacles to overcome when meeting with people from diverse backgrounds in the medical space. The second largest challenge is finding enough resources to accommodate the opportunities that arise after these groups begin to understand how our technology can be applied to their subset of industry challenges. We have not found the issue to be a stigma with gaming, but just in the realization of the tremendous potential that exists by harnessing the capacity and capability of the gaming community.
Can you tell me a bit about the successes HEWMEN’s already achieved in its Alpha stages?
One of the biggest results was the validation of our approach and techniques. Specifically, Eye in the Sky: Defender was able to show that after about 7 levels of play (~15-20min), players were analyzing the OCT images of Age-related Macular Degeneration (AMD) with the same level of accuracy as trained professionals. This showed that players engaged through HEWMEN were capable of analyzing images as well as providing insight on where the disease state was located. These results are being compiled and put together for a publication. The alpha event also allowed for the following publications as well:
2018 IEEE 6th International Conference on Serious Games and Applications for Health (SeGAH)
Proceedings of the 12th International Conference on the Foundation of Digital Games
The alpha event also provided analysis on our chemotherapeutic co-medications data which is used to help find treatments for multi drug resistant cancers. This is a collaboration between BALANCED and SMU’s Center for Drug Discovery, Design and Delivery group, led by Pia Vogel, PhD and John Wise, PhD. During the event streamers played a modified version of Minecraft BedWars. During gameplay they help identify properties of lab test compounds that can be used to increase the success of further compound selection. This event also helped foster a relationship between SMU and LSU Health in Shreveport, where compounds analyzed in during gameplay are being sent for animal trials.
What are your next plans for HEWMEN now that it’s released into its public Alpha?
We are working on putting together another event targeted around the end of 2018 to beginning of 2019. This will provide us with another set of data towards our ongoing research and development, but also help formalize partnerships we are creating with influencers, foundations, research centers and corporations. This event will lay the foundation to show the complete ecosystem of HEWMEN in action:
Allowing foundations and research partnerships to connect to the gaming community while having their processing and computational needs met;
Game community influencers having the ability to connect their community to causes and provide opportunities to help support the growth and engagement of their community;
Connecting industry partners to the large computational capability of the world’s first Super HEWMEN computer, along with providing opportunities to develop completely new applications that are now possible with the integration of humans and machines into a single platform;
And providing game developers a new opportunity to support their creations through the integration of HEWMEN, while simultaneously bring causes to their development and creative efforts.
Many thanks to Dr. Corey Clark and the wider BMT team for sitting down with us! With HEWMEN’s barreling release into an Alpha state and an incumbent release onto popular gaming platforms, it’s only a matter of time before we talk about it’s progress in linking gaming and medicine very soon.Related: AI, Alpha, balanced media technology, HEWMEN, Interview, medicine, Tech