JLJac on May 02, 2014, 05:56:06 AM (Last Edit: May 02, 2014, 06:01:26 AM):

Hm, ok, so here's what I'm trying to do:

A standard room is about 1040*800 pixels big. I want to be able to have a level that't at least three times that size as well, or variable room size isn't really worth it.

If I'm having variable room size, I need to have a movable camera, obviously. If I'm having a movable camera, I want parallax scrolling.

Each room consists of 30 bitmap layers (with very few colors) that has previously, in the static camera build, been baked together as one image at level generation. My dream was to in this new version have all those 30 layers move about as parallax scrolling layers when the camera moved. It didn't seem so impossible - if crysis can be crysis I can have 30 pretty big 2D sprites move around, right?

Because I wanted the dynamic shadows as well, I wanted all layers to be in one texture. That way I'd be able to check in say layer 28 if it's being shaded by say layer 27 by simply asking different coordinates in the same texture for whether or not they're transparent.

So I went ahead and stacked all the layers on top of each other in a texture, and 800*30 = 24 000, so that's considerably bigger than the limit.

And now I think I have to figure out some other way to do this haha   Maybe I'll have to drop the parallax scrolling after all, and go with some pre-baked solution - can can still save a depth map for the room graphic and use that to do the dynamic shadows, so all is not lost!

Edit: Oh, and yeah, I can save each layer as its own texture, but that would make each room a 32 file, 2 megabyte monster, so I'd prefer not to. My talk about a custom file format is because most pixels actually don't change from layer to layer, so if I wrote my own compression algorithm that worked on the z-coordinate I could probably get the filesize down some, but I'd still need to unpack it for unity, so it wouldn't really solve anything.

JLJac on May 02, 2014, 06:20:44 AM:

Hm, not really, it's an either-or situation, because the graphics consist of the 30 layers, and if I'm unable to displace them in the game I need to do it in the level editor, and I can't have two conflicting displacements or every line in the z-dimension would look jagged :/

Maybe I can make it happen through some sort of bump mapping-like shader...

JLJac on May 02, 2014, 06:34:50 AM:

It seems like a not very elegant solution, is all... Looking at this steep parallax mapping, it looks to me like it's a fragment shader which basically does a per-pixel raytrace both from the camera to the surface and then from the surface to the light source. Maybe something like that could be done here as well? If they have the time to step back and forth like that on each pixel, what I want to do shouldn't be impossible either, right?

http://graphics.cs.brown.edu/games/SteepParallax/index.html

JLJac on May 02, 2014, 08:26:49 AM:

Well, it's not really a question of large sprites moving about as much as one unified terrain that's just going to be slightly displaced according to the perspective of the camera - so if we wanted it to be done properly parallax scrolling would be the solution, but I think it could very well be solved using some kind of depth map and a shader as well. I.e it's more about objects being viewed from slightly different angles rather than being moved around.

But maybe the dynamic shadows will be enough to give a sense of depth to the scenes, maybe scrolling environments aren't necessary. I'll have to go over this in my head a few times before I settle on a decision.

JLJac on May 02, 2014, 09:19:50 AM:

This sounds super nice! And I don't quite understand what you mean - how does rendering as quads solve the problem with the texture being too big?

Dancing dead - Yeah I totally know how you traditionally do parallax scrolling, and that was my original plan! But when I ran into this hurdle I started thinking about the ray tracing shaders such as parallax mapping as another way to solve it, without actually storing all that data, but rather working from something like just a texture and a depth map. The parallax scrolling here isn't really distinct objects with large distances between them, but rather one single box might consist of 10 layers very close to each other, making it more of a voxel solution. So I though maybe I could fake that effect without doing parallax proper. Like, just bending the edges of stuff a little to fake some shift in angle, like this:

JLJac on May 02, 2014, 09:34:16 AM:

This is already how I do it! I just bake it together to a single image. All the rain world level art is 30 bitmap layers, each of them slightly smaller than the next, that together form semi-threedimensional objects. What I want to do now is the exact same thing, just at run time - so if I manage to do it it'll supposedly look the same, just moving.

Each layer texture has only a few colors, to indicate what color to grab from a palette. Maybe 6 colors all in all, including white which is treated as transparent. What's a tilemap? Can it help me?

JLJac on May 02, 2014, 11:25:02 AM:

  hahaha ok you guys are right, I've gone a bit overboard with this. I had a very particular vision, and an idea of how to make that happen, and when it didnt work I got a bit obsessed with making it work. I'll just look into other options, such as pre-rendering a few screen positions and switching between them with hard cuts. Don't worry, I've not become crazy obsessed with shaders and I won't mess up the game hehe

JLJac on May 03, 2014, 02:30:02 AM:

Ok, so I actually managed to mock up the thing I want - it looks a little something like this:



30 layers parallax scrolling voxel like 3D space renderer!

Even with the smallest level size and no shadows it still eats away at the frame rate pretty badly. I think I'll try to figure out some other solution.

Maybe the past two days make a little bit more sense now?
My original idea was to have all the layers in the same texture, and do a per-pixel raytrace layer for layer in the texture until I hit something. When loading a big texture proved impossible that original approach fell flat, and I've been trying to find another solution since. Now I'm pretty tired of all this though, and am contemplating to just have fixed screen positions. It would be nice to show the depth of the rooms like this, but you can't have it all. I still have high hopes for dynamic shadows falling across the geometry though - that could be done with just depth map and a pre-baked image.

JLJac on May 05, 2014, 08:49:56 AM:

Thanks guys! That's certainly an option, but the thing is that I already have this 30 bitmaps thing set up - that's how the level editor works already, except it bakes them together. Making it proper 3D would be an entirely different animal all together. I still have issues accepting that 30 sprites with somewhat high-res textures can be an actual issue when Crysis can do entire islands with a bazillion polygons of rain forest and stuff, but I'll take your word for it.

Update 132
So I had a talk with James, and we decided that before I got manically obsessed with making this tiny little parallax effect work, I could probably follow through on a fixed-screen approach. Especially as we'd most reasonably want both solutions in the game anyway even if I managed to make the parallax work, so we could fix the screen in performance heavy rooms and save the parallax for cinematic set pieces.

So, I'll abandon parallax scrolling for a while in favor of importing 2D backgrounds. In the final spasms of my parallax obsession, I gave the pixel-raytracing some time, and for a wile it looked bright because a 30 iteration for-loop in the fragment shader actually didn't murder the framerate as badly as I feared. But then some stupid stuff came in the way, namely that in Unity the CG compiler always unrolls all loops (stupid, but necessary on older cards if I got it right) and then if I do some operation in the for loop it assigns new temporary variables for each iteration (STUPID, NO EXCUSE) which maxes out the shader's tiny little working memory and keeps it from compiling.

So there I was, at a road fork, where one option was to learn the thing that graphical C compiles to (looks like this:
seems to be polish notation and similar horrors)  and the other was to decide that f*ck it, for now at least. So I'm trying to convince myself that #2 is the way to go.

I'll probably still get some good opportunity to pull my hair over shaders, if I'm going to do the coloration and the light stuff as a shader.

Before we leave the parallax issue, for now, I'll give you the the most current version of the "30 files, 30 sprites"-solution, to give you an idea of what could've been and might actually be if I get back to it later.

jamesprimate on May 05, 2014, 04:10:40 PM:

wow this looks like just the ticket

JLJac on May 06, 2014, 11:07:24 AM:

Thanks guys! Hope we can make this dream come true
And as always Jimbert, thanks super much! That might actually be a viable solution to some of our problems. I'll give the trial a test run eventually, when I get back to this parallax thing.

Hahaha Ulillillia is wonderful! I've never seen anyone care as little about the actual interactive layer in a 2D game 

Update 133
Ok, so despite encouragement and tips I've still managed to keep out of the parallax hole. Now I'm starting to set up the infra structure for the 2D level graphics instead.

My current idea is this - encode as much visual information as possible into a texture, and then unpack it with a pixel shader. If you guys want to tell me that's a horrible inefficient way of going about it, now is the time.

So, I started by setting up a list of the stuff I need to know for each pixel:

its color [0-3] (transparent, shadowed, standard, high light)

its depth [0-29] (Needed to give it the correct fog shade, and later to do the cool dynamic shadows I'm hyped about)

if it's lit [bool] (I'll still pre-bake the light, and save whether or not each pixel is actually lit. If it's not lit, we can save time in the shader by not doing shadow calculations on it.)

if it's a rainbow pixel [bool] (If you check the screenshots you might notice a little oily effect around the edges of stuff - that)

Whether or not it has an effect color [0-3] (No effect color, effect color1, effect color2, white. You know how some plants and stuff tend to be pink or cyan? That's this. White is used for the flies' hives.)

How much of that effect color [0-255] (probably unnecessary resolution here, if I make it 0-50 the gradients would probably still look smooth, given how they're always on small surfaces)

Then I started messing around, and realized that all of this information can be stored in a 24bit rgb pixel! With quite a fat margin, I could store a lot more if I wanted to!

So this is my plan:

Red channel:
0-transparent

1-30 = depths 0-29, color shaded, not lit
31-60 = depths 0-29, color mid, not lit
61-90 = depths 0-29, color highlight, not lit

91-120 = depths 0-29, color shaded, lit
121-150 = depths 0-29, color mid, lit
151-180 = depths 0-29, color highlight, lit

Green channel:
0 = no effect, not a rainbow pixel
1 = level effect color1, not a rainbow pixel
2 = level effect color2, not a rainbow pixel
3 = white color, not a rainbow pixel

4 = no effect, rainbow pixel
5 = level effect color1, rainbow pixel
6 = level effect color2, rainbow pixel
7 = white color, rainbow pixel

Blue channel:
0-255 amount of effect color

My hope and dream is that the level editor will be able to encode each pixel with this information, save it as a .png, and then a fragment shader in unity will be able to decipher it and color the level dynamically. Tune in tomorrow to see it shattered.

Unity wizards (like Jimbert!) can you give me a quick heads up on one thing - if I want to feed a shader a palette of something like 12 colors, what's more efficient? Feeding it as params, or feeding it a tiny little secondary texture where it can pick the colors?

JLJac on May 06, 2014, 11:21:47 AM:

All the more reason to fight to keep the pixels pure then haha! I'm a bit more worried that I've heard you shouldn't use if statements in shaders, and the decoding of this system is obviously going to be a huge knot of ifs and modulus operations...

jamesprimate on May 07, 2014, 01:55:50 AM:

ah yes! thanks for the link. David contacted us a while back to ask questions and if we'd be cool with him using RW as an example. which is awesome because that was a fantastic presentation, I learned a lot. Not as much as from Gimym's links though

JLJac on May 07, 2014, 02:27:52 AM (Last Edit: May 07, 2014, 03:51:22 AM):

Awesome! Thank you, Jimbert My implementation is actually very similar to those gradient maps, except my input is ... what would it be, 7 dimensional, and the output is a coordinate on a 2D palette texture rather than an 1-dimensional gradient.

I've had pretty good luck today, and got quite far with my deciphering algorithm! Then I ran into a weird error. I hit my instruction limit, which was kind of due to happen because the basic settings only allow for 64 instructions.

No biggie, let's just upgrade to shader model 3.0, right? That's like, 2003 graphic cards or something, people's PC's will be able to handle that.

No can do. For some reason Unity says "no subshaders can run at this graphics card" if I target the shader at anything higher than 2.0. Unity refuses to believe anything else than that my graphics card is from 2002.

I've searched Unity's settings for the "dude I bought my computer 2 months ago it's not ten years old"-checkbox, but can't seem to find it. There's an option to emulate for 3.0, in order to try the compability with older machines, but since Unity is adamant that my machine is even older than those older machines, no luck.

Any tips?

Found a command SystemInfo.graphicsShaderLevel in Unity, and this claims that I have 3.0, which is worrying for every reason. First, if I have 3.0 it should work, right? Second, I don't have 3.0, I'm supposed to have 5.0! Fun times!

JLJac on May 07, 2014, 04:35:29 AM:

James solved it.... Hahahahahaha! 

I was looking for high tech solutions all over, and the answer was this: Go to the "We now feature directx 11" page on unity.com - acquire the information that in order for complex shaders to work, you need to tick the "use directx 11" box in the unity editor. Tick the "use directx 11" box.

JLJac on May 07, 2014, 08:55:23 AM:

I'll write a fallback shader, so no worries

JLJac on May 07, 2014, 11:40:50 AM:

Update 134
A very good day!

I got a good chunk of my shader done - there is still work (dynamic palettes, effect colors, the rainbow effect, etc etc) but now I know that the basics work.

The level editor encodes an image looking as such:



This in turn is deciphered by the shader, which can tell things such as depth, whether or not the pixel is lit, and all that stuff I mentioned in the last post.



The shader is also fed another texture, a palette texture which is very small. In the future I think it might be cool to be able to interpolate between different palettes, to do cool effects such as darkening as the rain comes, lightning and the such.



This one has a little bit of random static in it - that will not likely stay, I'm just having fun.

And, finally, I got my dynamic shadows to work!



Woooho!

I have it working with the normal color scheme as well, just used this wacky palette so you'd see clearer. The principle is super simple: each pixel has a depth value (red in the mockup). If it's lit, it looks at a certain pixel to check if it's occupied by a sprite. Which pixel it looks at is determined by stepping backwards in the light direction.

So say that I'm a lit pixel that's at a depth of 5. Then I step, for simplicity's sake, 5 pixels to the left and upwards, and ask that pixel if it has something rendered on it. If true, I become shadowed. If I'm at a greater depth, I go more steps up and to the left.

It's unpolished - for example it doesn't take notice of perspective yet. If the shadows are cast straight backwards, you'd want them to stretch towards the center of the screen according to perspective, right, and that's not factored in yet. But I'm getting there!

Super exciting to see that it works!

jamesprimate on May 07, 2014, 04:41:11 PM:

 
 

JLJac on May 07, 2014, 10:42:12 PM:

No, I'm actually not raymarching, as I don't do any step-by-step checking of different depths. Instead the procedure has only two steps:

1. Check the pixel and get its info, which is color info but also depth (already encoded by the level editor).

2. Use the depth to check for where in the grab texture to look for objects that shadow this pixel. The greater the depth, the more up/left we check.

Effect: If you are in front of a wall, there will be a shadow on the wall. The further back the wall is, the further down/right the shadow will appear.

Your pseudo code is super helpful, by reading it I see a lot of things I can optimize in my shader! It's impressive that you manage to keep away from if-statements like that!

My texture looks like this:



So that's the same basic idea, right?

JLJac on May 08, 2014, 12:50:01 AM:

Thanks 

Ugh, can someone clue me in on the quirks of the CG compiler? Is there a list somewhere? It certainly does not execute the the code linearly in the way that I'd expect, that's for sure!

Example:


This code makes the lit areas green and the other areas purple, as expected. This though:


doesn't work, it executes the "else" always, never the "if" No other changes!

This:


works, and according to my logic that is the exact same thing as the one above it. Confused 

I've been told that shaders work more in parallel an less linearly, and I guess this is that, but I just can't wrap my head around how it works. To me it seems to just be random. What's going on?