The 4orge: Map optimization

Community General Discussion
1

Halo 4 will most likely have Forge, upgraded and new from Reach’s Forge.
Any competent forger will know that framerate is one of the main things to stress about whilst constructing a map.

People have been discussing what features they would like to see in Halo 4, from terrain editing to more variety in color, to resizable objects. While this is well and good, there is one thing I would like to see in the 4orge that is more behind the scenes. Map optimization.

In essence, the idea is simple, but it would be a HUGE leap forward in forges abilities.
To put it simply, after a map is complete and saved, the ‘optimization’ process will begins. The process finds all possible visible surfaces, and makes note of them. It then searches for hidden areas, such as merged objects and parts of objects that are below the surfaces of the map, and notes them separately. Then the process will save the map with all the unneeded polygons removed. This way, framerate and object density will become much less of an issue, and allow skilled forgers to boldly forge what no man has forged before.

Another feature that could run alongside this is a simple set of options (similar in Interface to the ‘Phased/Fixed/Normal’ options). The options would thus:
Reflective / Non-reflective
Opaque / Transparent
The non-reflective option will be a constant texture, not effected by lights. Great for optimizing framerate.
The translucent option will allow the object to be turned to glass.

These options should be available for a wide variety of objects.

Thoughts, opinions? Have I finally gone mad?
Feedback appreciated.

~ Der Flatulator6

2

This is pretty reasonable compared to other things wanted.

3

I’ve suggested such things in the past too.

You seem to be looking for a post construction processing tool. It’s probably reasonable possible but has to be part of the production plan from the beginning.

I’d also like to see a post process for applying proper light maps for anything that gets Forged. It would make maps so much better lit and looking. It would almost have to be a pay per use service. I understand light maps are a very resource intensive process. If it were even possible to set up such a service I’d expect prices like $10 to $20 bucks per map with no money back guarantee.

4

Actually, once you do the optimization, I can see it being a different map and some of the content lost in the process. Meaning, you “compile” the map to a production level map.

If the optimization is fast enough, and I suspect it could be, then it might make more sense to have it at game time when the map loads.

5

This is a great idea, but there is one problem. There is almost no way the game will be able to tell if a certain part of the map is visible to a player or not. Maybe the player could do this process themslevles?

6

> This is a great idea, but there is one problem. There is almost no way the game will be able to tell if a certain part of the map is visible to a player or not. Maybe the player could do this process themslevles?

If an object is phased inside another or inside the ground then you clearly can’t see it.
Not loading the invisible part would be, on its own, a great improvement to forged maps that rely on lots of phasing - which are generally the most visually complicated ones and thus those which suffer the most from frame rate-related problems.

7

Well, yes. In those types of situations it would, and that would greatly improve frame rate on its own. But, for example, there’s an amazing map set inside of a UNSC cargo ship. The map is built in the air and has pieces jutting out of the outside of the map. IN this situation, it would not help very much. That is why 343i should let the player do this process to anything that isn’t deleted automatically by the game.

8

those ideas sound awsome! but does anyone know if 343 putting forge in halo CE anniversary?

9

> those ideas sound awsome! but does anyone know if 343 putting forge in halo CE anniversary?

No, but you can forge on the 6 multiplayer maps within Reach.

10

This idea, if implemented, will backfire horrendously.

Loading a map in Forge without your “optimization” is as simple as loading, spawning, and moving several predefined objects.

Your idea would require the game to calculate entirely-new unified meshes, generate new UV maps to prevent visible texture changes, generate new collision meshes, and so much more – all of which is very complicated work. Editing this structure in Forge would probably be impossible. But the worst problem would be the loading penalty: instead of simply loading and spawning predefined objects, the game must now load the map as one entire 3D mesh.

File size is also an issue. Most map variants in the existing format never exceed 700KB, yet the File Browser still can’t efficiently operate on them in bulk. With your “optimization”, the map variant files themselves would become absolutely massive compared to how large they are currently, and the already-inefficient File Browser would become literally inoperable.

OP, your idea completely breaks saving, listing, and loading. Considering how pathetically inefficient those latter two operations already are, Reach simply can’t afford anything that would impair them further.

EDIT: To clarify: the existing format seems to just be a list of object names, positions, and user-set properties. Your optimization, OP, would require the storage of 3D mesh data, UV maps, etc., in the map variant file. That is a considerably large amount of data.

11

> This idea, if implemented, will backfire horrendously.
>
> Loading a map in Forge without your “optimization” is as simple as loading, spawning, and moving several predefined objects.
>
> Your idea would require the game to calculate entirely-new unified meshes, generate new UV maps to prevent visible texture changes, generate new collision meshes, and so much more – all of which is very complicated work. Editing this structure in Forge would probably be impossible. But the worst problem would be the loading penalty: instead of simply loading and spawning predefined objects, the game must now load the map as one entire 3D mesh.
>
> File size is also an issue. Most map variants in the existing format never exceed 700KB, yet the File Browser still can’t efficiently operate on them in bulk. With your “optimization”, the map variant files themselves would become absolutely massive compared to how large they are currently, and the already-inefficient File Browser would become literally inoperable.
>
> OP, your idea completely breaks saving, listing, and loading. Considering how pathetically inefficient those latter two operations already are, Reach simply can’t afford anything that would impair them further.
>
>
> EDIT: To clarify: the existing format seems to just be a list of object names, positions, and user-set properties. Your optimization, OP, would require the storage of 3D mesh data, UV maps, etc., in the map variant file. That is a considerably large amount of data.

I hate it when people on these forums talk how everything is impossible to program…
I don’t know your programming/engineering background, so sorry if you actually know what you’re talking about, but it just pains me to see that every time a thread is created about ideas for future features, there’s always at least one post saying “it’s impossible”.

OK, rant aside…

This is not a discussion about Halo: Reach, it’s about Halo 4, and we all hope (because we are ignorant like that) that 343i has created a new, improved Forge Engine and File Handling System that can support features that the current ones can’t handle.

I certainly don’t know how it’s done, but think about all of the base maps (not created in Forge) and the Campaign maps and the Firefight maps. I seriously doubt it’s one single 3D mesh. Well, I’m actually 100% sure it isn’t because we saw in a ViDoc some employee adding/un-hiding vegetation and trees with a single click.

Optimization doesn’t necessarily mean creating one single 3D mesh - you just need to slice the Forge Objects and simply not render the parts that are invisible or phased inside another object, all done when the map loads (so the objects are still there for you to re-Forge).
If you think it’s impossible to dynamically slice objects then you are wrong (just like anyone who says something is impossible to program). Take a look at Metal Gear Solid Rising. Now imagine this but without all the graphical glitter and multiple child spawning for physics treatment.

Seriously, if they want to do it they can - even if it doesn’t make it to Halo 4, I hope it will make it to Halo 5 or 6. Or maybe something entirely different and gloriously better.

Sorry if I wasn’t very clear - it’s late…

12

> This is not a discussion about Halo: Reach, it’s about Halo 4, and we all hope (because we are ignorant like that) that 343i has created a new, improved Forge Engine and File Handling System that can support features that the current ones can’t handle.

Ah. My mistake, then, regarding the File Browser part of the problem.

> I certainly don’t know how it’s done, but think about all of the base maps (not created in Forge) and the Campaign maps and the Firefight maps. I seriously doubt it’s one single 3D mesh. Well, I’m actually 100% sure it isn’t because we saw in a ViDoc some employee adding/un-hiding vegetation and trees with a single click.

Correct.

> Optimization doesn’t necessarily mean creating one single 3D mesh -

Correct.

> you just need to slice the Forge Objects and simply not render the parts that are invisible or phased inside another object, all done when the map loads (so the objects are still there for you to re-Forge).

There are two ways to do this.

The first is to simply determine which polygons are not visible to the player. A simple rendering optimization, in other words. This is already done in every Halo game’s engine, and probably in every modern game engine.

The second is to literally remove unseen faces from the geometry. This seems to be what the OP was talking about (“Then the process will save the map with all the unneeded polygons removed.”), and it has every single problem that I listed in my post.

> If you think it’s impossible to dynamically slice objects then you are wrong

That’s not what I said. All I said is that the optimization would only serve to slow down the saving and loading of files, while also bloating file sizes. The only possible benefit would be in terms of rendering, and that benefit can’t even be proven. (That is, we can’t prove that the deleted polygons wouldn’t have just been skipped by previously-existing graphics engine optimizations.)

> (just like anyone who says something is impossible to program).

Thinking that everything is possible is even more irrational and logically backward than thinking that everything is impossible.

> Take a look at Metal Gear Solid Rising. Now imagine this but without all the graphical glitter and multiple child spawning for physics treatment.

Swordfighting and, by extension, object slicing is one of the major, core gameplay elements in Metal Gear Solid Rising. It stands to reason that the game engine was specifically built from the ground up with object slicing in mind. Halo 4’s engine, on the other hand, is not built from the ground up with object-slicing in mind. (It’s a first-person shooter, so why would it be?)

Game engines are not nearly as similar to each other as you seem to believe. Comparing the Halo engine to something like Metal Gear Solid Rising is like comparing a microwave to a stove. They’ll both heat things up, but that doesn’t mean you can just take parts from one, put them in the other, and expect the device to work smoothly (or at all).

13

> > you just need to slice the Forge Objects and simply not render the parts that are invisible or phased inside another object, all done when the map loads (so the objects are still there for you to re-Forge).
>
> There are two ways to do this.
>
> The first is to simply determine which polygons are not visible to the player. A simple rendering optimization, in other words. This is already done in every Halo game’s engine, and probably in every modern game engine.
>
> The second is to literally remove unseen faces from the geometry. This seems to be what the OP was talking about (“Then the process will save the map with all the unneeded polygons removed.”), and it has every single problem that I listed in my post.

Well, in that case it might not be a smart move to do, because after deleting the polygons you can’t move the object in Forge and expect them to reappear.
I don’t think that’s what the OP meant. I don’t think the OP wants something so specific to happen - maybe he doesn’t even care -, I believe he just wants something like that to happen, some feature that tells the engine to not care about certain polygons.
What we’re doing here, now, is coming up with ways to actually do it.

Getting rid permanently of the undesired polygons isn’t, again, smart or something we should desire.
That has problems beyond what relates to performance - it directly affects the player, the forger, by obliterating parts of the Forge objects he used.

What was theorized here was temporary obliteration, done on game time. All the information, every single polygon of the objects used is in the map file - the beauty of it is that when you play the map outside of Forge, the engine determines which polygons are expendable and then either doesn’t display them (you can’t see them but they’re still there for collision detection, lighting rendering and all that. This might be the least favorable option.), stops caring about them (doesn’t display them and doesn’t compute ANYTHING for them - co collisions, no lighting, no nothing.) or completely removes them from the parent object (which, in a way, is pretty much “not caring about them” but it costs more resources to achieve because it actually has to destroy polygons, which doesn’t happen when it “doesn’t care about them”).

The optimum solution is not caring about the polygons - they’re there, but ultimately they’re not.

> > If you think it’s impossible to dynamically slice objects then you are wrong
>
> That’s not what I said. All I said is that the optimization would only serve to slow down the saving and loading of files, while also bloating file sizes. The only possible benefit would be in terms of rendering, and that benefit can’t even be proven. (That is, we can’t prove that the deleted polygons wouldn’t have just been skipped by previously-existing graphics engine optimizations.)

That’s true if you create a single super 3D mesh.
If you “don’t care about the polygons even though they’re there” then that’s not a problem, especially because it doesn’t interfere at all with the map file - it’s just a single, one-shot computation done every time a game starts.

> > (just like anyone who says something is impossible to program).
>
> Thinking that everything is possible is even more irrational and logically backward than thinking that everything is impossible.

No. You can program anything you want (HURR CAN’T PROGRAM LIEF ORBITING AROUND A 30ºC STAR MADE OF PONIES). If you can’t, then someone can. If there is no technology for that, you must invent it - either by creating a more powerful programing language (or just simply not using C++) or creating new hardware or creating anything else.

Everything is possible. (R)

> > Take a look at Metal Gear Solid Rising. Now imagine this but without all the graphical glitter and multiple child spawning for physics treatment.
>
> Swordfighting and, by extension, object slicing is one of the major, core gameplay elements in Metal Gear Solid Rising. It stands to reason that the game engine was specifically built from the ground up with object slicing in mind. Halo 4’s engine, on the other hand, is not built from the ground up with object-slicing in mind. (It’s a first-person shooter, so why would it be?)
>
> Game engines are not nearly as similar to each other as you seem to believe. Comparing the Halo engine to something like Metal Gear Solid Rising is like comparing a microwave to a stove. They’ll both heat things up, but that doesn’t mean you can just take parts from one, put them in the other, and expect the device to work smoothly (or at all).

Again, this just proves you can do it.
Sure - it needs work.
Sure - might not be possible to invest that work in a project that’s almost finished.

This is a Forum. We are blindly suggesting feature we think might be beneficial and none of us expects them to be considered.

Hell, I’m 100% sure the guys at Bungie thought about something like this.
I’m almost certain such a system has been created and maybe Halo even uses it!
We’re probably discussing things that have been fully developed and tested and implemented years ago…
But that doesn’t really matter, does it? :slight_smile:

14

Post deleted.

15

Something that would possibly work is to split editable files and published files. After running through an optimization process like the one described in the OP would render the map completely uneditable… in a “published” state. The same way that when you are done making an image with a ton of layers in photoshop, everything is combined into one and is used to create a single published image. The editable version would be the smaller files with just the object IDs, coordinates, and rotations. The published version would be fully optimized for efficient rendering, but as a result would have a larger file size. It would recalculate lighting, remove unnecessary polygons, etc, etc. The number of published maps would be few and far between. It would basically be about the ratio of the number of maps on your hard drive versus the number of published maps that you have on your fileshare. I know for me that is a very large ratio… lol.

The real question is whether or not the size of the published file would be manageable enough or feasible for loading maps and sharing them across the internet. Keep in mind that anything you create needs to be shareable to other users. A massive file size may result in a VERY long pre-loading process for a map.

16

> The real question is whether or not the size of the published file would be manageable enough or feasible for loading maps and sharing them across the internet. Keep in mind that anything you create needs to be shareable to other users. A massive file size may result in a VERY long pre-loading process for a map.

This, definitely. This is my primary concern.

But again – we also don’t know if such an attempt at optimization would be worth it. Much of the things that have been discussed are already done in the engine (i.e. hidden surface determination).

17

> Something that would possibly work is to split editable files and published files. After running through an optimization process like the one described in the OP would render the map completely uneditable… in a “published” state. The same way that when you are done making an image with a ton of layers in photoshop, everything is combined into one and is used to create a single published image. The editable version would be the smaller files with just the object IDs, coordinates, and rotations. The published version would be fully optimized for efficient rendering, but as a result would have a larger file size. It would recalculate lighting, remove unnecessary polygons, etc, etc. The number of published maps would be few and far between. It would basically be about the ratio of the number of maps on your hard drive versus the number of published maps that you have on your fileshare. I know for me that is a very large ratio… lol.
>
> The real question is whether or not the size of the published file would be manageable enough or feasible for loading maps and sharing them across the internet. Keep in mind that anything you create needs to be shareable to other users. A massive file size may result in a VERY long pre-loading process for a map.

This was what I was saying with “compile” into a new production map.

The issue would be load time. If you try to load a map, you will see that the load time is the canvas, not the map data. So this would load just like any other canvas. This is why when you load FW map A it takes a while, because it is loading the FW canvas. Then you load FW map B, it is instant, because the canvas is loaded. Then you load Anchor 9, and it takes a while, as it must load a new canvas. Then you load FW map B, it takes a while to load FW canvas again. I doubt it would take any serious time to load a canvas we make. But I doubt we will ever see this type of compiler for Forge.

But you are correct with loading across the internet. To do this would be like DLC. But this wouldn’t be much a problem, because you could send the compiled components to an existing canvas, making the download short in content.