Archive for the ‘Silverlight’ Category

Silverlight 5 – The moon interior world

While looking at the moon as presented in the sample application, I thought: “wouldn’t it be fun to be able to get into it and then find a friendly landscape to explore”. What we need for this is a so called Sky Sphere, and a moving camera.

Sky Sphere

A Sky Sphere consists of a cube which faces are textured with a coherent set of 6 images. For instance, the 6 connected images resulting from taking a photograph into each of the 6 orthogonal directions, starting at the horizontal direction facing north. Within this cube is a sphere, and the inside of this sphere is textured with the cube textures by means of extrapolation of positions on the sphere onto positions on the cube. Then, within the cube you see the seamless projection of the cube’s images. This technique is used to provide an environment; it is also called environment mapping. The idea is to move the sphere and cube with the camera so the sky sphere always presents a distant environment. In the current version we have not yet set the step to always keeping the environment distant. At this point we want to be able to travel through the moon, to, into, through, out and away from it. The XNA Sky Sphere Community Sample served as a starting point for development.

Camera

A camera is in fact a component that provides you with a View matrix. There are a number of cameras available through XNA community samples. I explored:

  1. The ArcBall camera
  2. The Orbit camera
  3. The Chase camera

The ArcBall camera

The advantage of the ArcBall camera is that it has a large number of properties and many degrees of freedom. It therefore has a broad spectrum of applicability. On the other hand, the code is complex, and in any particular application it holds a lot of code you don’t need. Also it is hard to keep it leveled straight. Somehow I always managed to get the view somewhat tilted, but never managed to get out of the tilt. So I looked a bit further.

The Orbit camera

The Particles XNA community sample contains code for a typical orbit camera. An orbit camera can move around its LookAt, even zoom into it. This camera lacks the capability to move around freely, that is, to change, or control, the LookAt. So, we will need not one, but two camera’s, one of which will be this orbit camera.

The Chase camera

The Chase Camera is another XNA community sample. In the sample the target is a ship. In our project we do not have a visible target. The target is just the LookAt. Typically, the target, or LookAt, is what you control with the keyboard or game controller, the camera just follows the target. The target can also be controlled using the mouse. Instructions for controlling the chase camera can be found in the section “Sample Application” below.

What you control defines the major difference between a chase camera and an orbit camera. In the former case you control the target, and in the latter the camera (while the target remains stationary). One could easily create an integrated camera that switches between orbiting and chasing. For now, I’ve adapted the chase camera to fit the needs of this application, as described above.

Creating the Silverlight 5 version of phase 2

To port software from XNA to Silverlight 5 does (still) require real work. The first task is to split up the Effect file into separate shaders. One file for each vertex shader and one file for each pixel shader. Since we have an inside and an outside to draw, we end up four shader files that have to be loaded separately. The same holds for the .dds file that holds the image set for the environment map. The set had to be split into 6 separate file that have to be loaded individually.

At first I couldn’t find a way to load the Silverlight 5 textureCube into the shaders. But at some time I found that the TextureCube is a subclass of the Texture class, hence can be assigned to the textures collection of the GraphicsDevice which will then load it into the shaders at runtime.

The UI thread and the Composition thread of the DrawingSurface

We want to control the pitch (vertical angle), yaw (horizontal angle) and thrust by means of keyboard (and mouse, but see below). So, mouse and keyboard events set a variable that indicates that a change in angle is required. The mouse and keyboard events happen on the UI thread. The Draw event happens on the Composition thread. In order to prevent synchronization problems, we want to prevent (re)setting the yaw and pith variables on this thread.

In order to prevent both synchronization problems and loss of performance as a consequence of using locks or wait states, we set e.g. the pitch variable in the KeyDown event handler (for the Up and Down keys), and reset it in the KeyUp event handler. The Draw event handler only reads (samples) this variable.

In the RC (release candidate) version of Silverlight 5, the DrawingSurface and e.g. the Rectangle do not raise keyboard events. The Textbox, however, does. On the other hand, the DrawingSurface and the Rectangle do raise mouse events, but the Textbox doesn’t. Also, these events do not seem to be routed events, that ‘bubble up’, yet. So we have to choose between mouse input and keyboard input over the DrawingSurface. For this application I have chosen keyboard input. We use a transparent Textbox, stretched out over the DrawingSurface to raise keyboard events.

Keyboard control has its problems:

  • Before the TextBox has the focus, we first have to click it. We use a startup message that has to be clicked away as a work around.
  • It might well be that the left and right arrow keys respond only after a while. This does not hold for the up and down arrow keys, or the space bar. Reason unknown.

Remark on the render system: at the edges of the drawing surface, the shape of the moon gets distorted – flatted out to an ellipsis. This turned out also to be the case for the XNA version; something to look into.

Sample application

A sample application can be found here, as the Phase 2 app. The keys for navigation are:

  • Space: forward
  • Left (Right, Up, Down) arrow: rotate to the left (right, up, down)
  • T key: turns the camera around (180 degrees)
  • R key: resets
  • A key: toggles physics

The sample application demos the moon interior and the chase camera. And that’s it.

Edit 29-12-2011: Since the Release of Silverlight 5 RTM some changes were made to the sample program. There is no startup message any more. On the other hand, the application only runs after local installation.

Advertisements

Silverlight 5 – Rotating Moon in 3D

Providing 3D user experiences over the web is an extraordinary technological feat. It is possible when we use the XNA integration with Silverlight 5 (beta). This blog post describes a first experiment: showing a rotating moon. In upcoming posts I hope to extend this experiment to include some further nice features.

Approach

The approach taken here is to first develop a model in XNA, for PC, and to then port it to Silverlight 5 when finished. This provides access to well known and well developed coding idiom, and the means to test and debug the code. It also provides the opportunity to separate off any problems that are specific to the integration of XNA and Silverlight.

Starting Point

The starting point was the “Textures and Colors” sample from the Microsoft App Hub site. This sample provides the essential techniques for importing a sphere and texturing it with an image. The sample provides much more, but we don’t need that right now.

The mapping of an image, which is usually rectangular or square, onto a sphere is not straightforward. See the screenshot from the “Textures and Colors” Sample below (left) which maps a “Cloud” texture onto a sphere, and an experiment in which a quilt design was mapped onto the sphere.

There are two obvious ways around this problem. One is to have an image that has been prepared to compensate for this effect. The other way is to map the image to the sphere while compensating for its shape.

When we apply a technique called Spherical Mapping, we get somewhat improved results, but still not satisfying. Spherical Mapping uses Normal vectors to correct the u and v coordinates that are used to sample a texture for a given xyz 3D coordinate. Coordinates are calculated as:

u = asin(Nx)/PI + 0.5

v = asin(Ny)/PI + 0.5

where NX is the x-component of a Normal vector. Results in our cases are as follows.

The blue and white quilt ball looks much better, the cloud texture map still looks awful. Luckily, I was able to find a cylindrical projection map of the moon on the internet which looks quite good.

At first, I thought that the terrain was looking odd because of deformations caused by mapping the image onto the sphere, but this is the actual south pole of the moon. And, by the way, this is a much better image than the one you get from Google Moon (Earth).

Creating the Silverlight 5 application

The idea is to use the new DrawingSurface control to render the 3D scene. Needed here is a way to import the sphere model into a Silverlight 5 Application, and to use vertex and pixel shaders to position and texture it with the image.

Some people have already created practices and extensions that can be used to get this done. Entry point to these pioneers is Aaron Oneal’s Blog. Notable contributions that have been made are the CodePlex Babylon Toolkit project by (predominantly) David Catuhe, and The model loading CodePlex project by Simon Ferquel. We will use aspects of those contributions where required.

I’m somewhat reluctant to jump at CodePlex products, since experience taught me that frequently there are some disadvantages attached. For instance, a feature I really would like to use is the ShaderBuildTask. This automatically builds the shaders when you build your project. However, for the ShaderBuildTask the shaders should be attributed as ‘content’, not as ‘resource’. If not, you’ll get a build error stating that the output already exists. The disadvantage is that you cannot use the resource addressing idiom (the “;component/” format). Then the shaders cannot be loaded since the code that loads the shaders is in another Visual Studio Project than the shaders. So, back to the scripts from the first Microsoft Silverlight 5 3D demo application. See what I mean?

Anyway. The model Importer by Simon Ferquel did prove its use.

Sample App

A Silverlight 5 sample application showing the moon in slow rotation can be run at my App Shop. This is a separate corner of the App Shop: an HTML 5 site, presenting Silverlight 5 Apps.

Glassy Buttons on a Magnifier

Glassy buttons, or other controls, have a great appeal because of their transparent and shiny character. They remind us of gems. In user interfaces, the transparent character is also of great use in case you need to see what is beneath the control.

Recently, I dug up the Silverlight Glass Button tutorial by Martin Grayson. This glass button consists not only of a very fine looking exterior, but also has sophisticated animated behavior. An overview of the steps to create such a button is as follows below.

Creating the Glassy Button

  1. Create project in Blend
  2. Set the root background to a gradient color that blends well with the color of the glass button.
  3. Create a button and start editing the template (empty template option)
  4. Add a Border control to the template: the button’s Outer Border
    1. Add a second border, within the outer border: the Inner Border. Set the backgroundColor to Black, and the Alpha to 0.5.
    2. Add Grid to Inner Border with a row divider to middle of the grid
    3. Add border to top row: the Shine. Remove the margins, set the V / H alignment to stretch, and set the background to linear gradient from top to bottom. Set the Black gradient stop to Alpha = 60%, color=white. Set Other gradient stop to Alpha = 20%.
  5. Add ContentPresenter to the center of the Button. Set it between Shine and Glow (see below). Set Foreground color to white (or other color that stands out).

This is the standard look of the button. Now add a blue Vista glow for the MouseOver event.

  1. Add another border, called Glow to the Grid, RowSpan = 2. Width=Auto, H/V align = stretch. The Background is a RadialGradientBrush. Set the left gradient stop to a light blue (141, 189, 255) with alpha: 70%, and the right stop to the same color with alpha = 0. Expand the gradient beyond the bounds of the button to look like the rising sun in the top row. Set the glow behind the Shine. Set opacity of the control to 0, animation that will show it will be added later
  2. Animate the MouseOver state transition by moving the Glow opacity in 0.33 s to 1.0, including revert.
  3. Animate the button Press state transition by hiding the Glow (Visibility:Hidden), setting the Opacity of the Shine to 40%, and settting the Background Border opacity from 80% to 50%.

That’s it. Of Course, the color of the button is arbitrary, as is the color of the Shine and Glow. This idea could be extended by providing an extra bottom layer onto which you could project items. Possible items are an image, a video, a Dropshadow of the text in the button’s Content control, etc. Another extension might be to apply the inner shadow by Samual Jack.

The Magnifier control

The glassy button has been used in a magnifier control. A magnifier can be used to enlarge pieces of the client window in order to study it with more precision. This specific magnifier has five buttons: enlarge what is under the magnifying glass, reduce it, reset it to normal, and then: flip the image horizontally or vertically (toggles). This last functionality may come in handy if you want to study rotated / mirrored images.

Note that this is the second time that we encounter the need for a jog / shuttle control: to be able to rotate the magnified image to arbitrary angle, apart from mirroring, would be an improvement. In this specific application we would use the jog / shuttle control as a rim around the magnifying glass.

Okay, back to the Glassy Buttons. They fit nicely in this design since you can look through them when positioning the magnifier. The outer border of the magnifying glass has been transparently colored using a rainbow like gradient so you can find it back, also in a multicolored image like the one in the example App.

In order to position the magnifier, you can drag it around by the rim. The functionality was added using the Expression Blend MouseDragElementBehavior. Although this behavior doesn’t work for Buttons, it works for controls that contain Buttons.

The magnifier uses two WriteableBitmaps. One to hold an image of the LayoutRoot, and one to hold the image of the magnifier. Copying is done using the Clear and the Blit function in the WriteableBitmap Extensions. Of course, before copying the visible rectangle has to be determined, for situations in which the magnifier is only partly visible (off screen / covered).

A specific problem solved was how to initially get the bitmap of the LayoutRoot. If the image is loaded from the hosting web site, and is not present in the xap itself, the LayoutRoot bitmap remains empty, until created after the first rendering. For all clarity, the creation of the LayoutRoot bitmap is located in the SizeChanged event handler. So, we have the initial LayoutRoot bitmap created the first time the magnifier bitmap is created – code guarded by a Boolean flag :-(.

The magnification itself, as well as the horizontal and vertical flips is done by means of a simple pixel shader.

Glossy Button

The MS Expression Blend tutorials contain a tutorial on creating a glossy button. Well, I like shiny things, so I got into it. The tutorial is very practically oriented, so in this blog post we’ll start with some remarks on lighting practices that might create the illusion of a shiny object on your screen.

Lighting

With respect to lighting concepts, I will somewhat follow Frank D. Luna here. Factors at play in lighting are directional and wavelength attributes of the light(s) involved and reflective properties of the material the light.

Light

For people, colored light is a compound of Red, Green and Blue light in varying intensities. The relative intensities of these components define the colors we know.

Ambient lighting is lighting of an object by indirect light, light that has been reflected multiple times, and thus has no specific source. In Ambient lighting models, neither the position of the observer nor the position of the light source plays a role. You might think of ambient light as showing the color of an object (dependent on the color of the light, of course).

Diffuse lighting of an object describes the scattering reflections of light coming from a specific source. The smoother the surface, the less diffuse its reflection, the shinier it looks. In Diffuse lighting models, the position of the light source, but not the position of the observer is defined. We could say that diffuse lighting creates a gloss on smooth surfaces.

Specular lighting of an object involves directed light that reflects (mostly) in a specific direction – ‘a cone of reflection’. In Specular lighting both the position of the light source and the observer is relevant – the observer might not see the specular reflection. We might say that specular lighting creates the shine on a smooth surface.

Light sources come in three flavors: parallel light sources, like the sun; point light sources, like a light bulb, and spotlights, like a flashlight.  In 3D models, parallel light has a direction but no source location; light from a point source has a source position and an
intensity that decreases quadratically with distance from the source. It lights objects in all directions. Spotlights have a source position, a specific direction and also attenuation. According to Lambert’s Cosine Law, reflected light intensity depends as the cosine function on the angle at which the light hits a surface, so perceived reflected light intensity depends on both the angle and the distance.

Material

Material properties define which wavelengths will be absorbed and which will be reflected (and to what extent). This defines the color and smoothness properties. Rough material will reflect light diffusely in all directions. If subjected to a strong light source, it will glow brightly, but will have hardly any gloss, and no shine. Conversely a smooth object will have a clear gloss and almost act as a mirror for the specular light (for an observer in an adequate position).

Modeling lighting of a glossy button

In a DirectX or XNA application you can model lighting extensively, and the result will be quite realistic. This realism comes at the price of significant resources, which cannot be spent on ‘just’ a button, so the same effects will have to be simulated by other means. In this section we will first analyze the construction of the glossy button from the tutorial, and then add some more features – for fun and enlightenment.

The Glossy Button Tutorial starts out with an ellipse, the button, that has a gradient color (and a robust edge). The gradient thus covers both ambient and diffuse lighting. A second layer, the gloss, consists of an ellipse that has a white gradient color, running into transparency. It also has a mild blur effect. This ellipse covers about two thirds of the button. The final layer, the shine, is a third ellipse, a much smaller one that also has a gradient white color, moving into transparency. The result is pretty cool, isn’t it?

Had I already mentioned the drop shadow (bottom right)?

Well, although pretty cool, one may have some disturbing questions, like: ‘What exactly is the shape of this button?’, or, ‘Where does the light come from?’ The shape cannot be a sphere, for that the gloss extends too far to the bottom, or alternatively, the shine is too close to the top. Also, it is weird that the drop shadow is bottom right, while the shine is in the middle. I also do not think that this button is very shiny, it doesn’t look it has a top layer of glass, like really shiny things do (I will get into this really shiny stuff in another blog post). Finally, it is strange that the rim has the same color all around the button. So, the conclusion is that the ingredients may be there, but the recipe isn’t quite right.

I have tried to improve a bit on these shortcomings but in the eyes of the reader it might just as well have become worse, so read on ;- ).

The glossy button demo application

The demo application has a number of additional features, the simplest being a text on the button. Also, the size of the button is rather large here, but it can be set differently using Width and Height dependency properties without adversely affecting the visual properties or behavior of the control.

Color picker

The user can pick a color for the button using the color picker. The color you select is the bright color in the gradient. The application adds the dark end of the gradient itself. The rim is also set to the dark color derived from the selected color. The color picker control is part of the CodePlex Silverlight Contrib project.

Follow the mouse pointer

Although I’m not a fan of things on your desktop that follow your mouse pointer, I did provide the option here. It was either follow the mouse pointer or develop a kind of jog/shuttle control to move the three gradients over the surface of the button, and this is quicker. So what you see is that within a certain range around the button, the button gradient, the gloss and the shine all follow the mouse pointer. The button does seem to a spherical shape when you move around.

Proximity color effect

A more experimental addition is that when the mouse pointer comes close to the button, light intensity increases. This means also that for some lighter colors, the color changes if the mouse pointer is over the button. This would agree with the common experience that harsh white light dims or removes color.

Mouse Pressed visual state

When the left mouse button is pressed, a simple animation increases the size of the gloss and shine.

Concluding remarks

After some pondering about some nagging dissatisfaction with the results I realized that a really shiny button also reflects the objects nearby. Reflection is what really makes the difference. So, if you really want to have shiny buttons, you are in for true 3D models in you user experience. Although that seems a far cry, the general use of 3D modeled user interfaces seems to gain momentum now. Just note tendencies like the use of the Kinect, the integration of Silverlight and XNA, the use of 3D in CSS3 and Html5, and, of course 3D video – without glasses even.

On the other hand, shiny buttons that do not reflect are already part of the Apple UX. In that case you see that the buttons are more like colored glass. These subjects, glassy buttons, and 3D user interfaces, will be subjects of upcoming blog posts.

The Windows 8, HTML 5, and Silverlight Rumor Circus

In this blog post an overview of the recent wave of fear and anger across the internet concerning the future of .Net in Windows 8 (could be released Autumn 2012), and why it is all a storm in a teacup.

Where and when it started

It all started with the demo of Windows 8 on the AllThingsD Conference (June 1)by Julie Larson-Green and Steven Sinofsky who mentioned that the applications presented were, and further applications could written in Html5 and JavaScript (version 1.8.5, together with CSS 3.0 called the HTML technology stack). Throughout the demo there was no mention of Silverlight or WPF. “This, What Has Not Been Said” tapped into already slumbering fears that HTML5 will compete out Silverlight. Heated reactions followed in discussion groups. See e.g. this one, where the thread was closed by the moderator. Some people, clearly driven by a distinct dislike of the Microsoft company stirred up the fire, as did a journalist of a respected medium.

What it is about

The fear mentioned above is the fear of many developers that costly investments of time and effort will become useless with the release of Windows 8. Of course, if there is only the risk that .Net software would be legacy at the release of Windows 8, investments in .Net and Silverlight software would stop immediately. Not only is there fear, but also dislike. Some developers express the opinion that the HTML5 tech(nology) stack is inferior to Silverlight, and also tedious to work with.

Does it seem justified?

I myself doubt this fear and uproar is justified. The HTML5 tech stack consists of ‘standards’ that are not finished, and have implementations that diverge across browsers, thus forcing web application developers to provide multiple implementations of the same functionality – who would like to pay for that? I know people that sell over the web and implement their web shop in HTML version Long.Ago to guarantee broad accessibility; this is what the Browser Wars have accomplished – people do not like to invest in new versions of the Html tech stack. Microsoft will not make itself dependent on these ‘standards’ it does not control.

Furthermore, Silverlight can do things that the HTML5 tech stack in itself cannot, or never will be allowed to do, if only for security reasons, see e.g. Microsoft’s refusal to support WebGL). Would Microsoft suddenly turn around, embrace this technology and replace its own? Unlikely.

However, Microsoft also didn’t move to take away the fears; they have the Build Conference in September 2011 at which they will tell more. Nevertheless, one might expect some indirect damage control, and it came quickly.

Damage control – differing opinions

First there was the blog post by David Burela referring to analysis of a leaked early Windows 8 build. The conclusion seems to be that software that will be built on top of Windows 8, will be built with .Net and Xaml. A particular group of applications, called ‘immersive’ applications (running within the Windows Shell) can be build using Html5 and JavaScript, much like some types of applications for Vista.

Mary Jo Foley of ZDNet goes further and publishes parts of correspondence with developers who have actually analyzed the early Windows 8 build. The picture that arises from that article is that an improved version of the .Net runtime (referred to as the Windows Runtime) will be central in Windows 8, and is programmable by Xaml in concert with a wide variety of programming languages, i.e. it is like Silverlight / WPF. Within Windows 8, XNA can be used for 3D graphics. Windows 8 apps built using ‘.Net’ should be easy to port to other devices – phones, tablets, etc. say by recompiling and compensating for the form factor. The Html5 apps might depend on the Windows Runtime as well (MD: this would explain the little understood remarks from Microsoft about native support of Html5). WPF and Silverlight may cease to exist as such in Windows 8, but the constituent technologies will be there.

In conclusion, it seems as if Microsoft is creating the facilities to build apps in Windows 8 using the HYML5 tech stack, as an addition to the .Net framework, rather than as a replacement. The motivation to do so, by the way, is to attract more, new developers to the platform. It does not seem to be the case that ‘immersive’ application can be build only in Html5. It seems that ‘Immersive’ is just a namespace, defining an API that is required to build applications which run within the Windows 8 shell.

C++

OK, so software can be built using a variety of .Net languages, among which C++, and the runtime seems to be closer to the metal, thus providing higher performance, because immediate OS layers have been removed.

But there is more. C++ seems to be in what is called a ‘renaissance’. More developers use it in order to gain higher performance, a new specification (C++0x) is on its way, and C++ is recently declared by Google to be the best high performance programming language.

For the next version of Visual Studio (also to be released in 2012) Microsoft announced the AMP Accelerated Massive Parallelism library at the AMD Fusion Developer Summit. AMP promises to provide full C++ access to a heterogeneous set of processors and their memory models. That is to say: you write one program that executes both on a computer with GPU, as well as one without it. Note that GPU’s are not considered to be restricted to rendering graphics. These people consider a GPU a broadly applicable parallel processor (and indeed, there exist Graphics cards without a monitor connector). The demos shown by Sutter and Moth reflect awesome performance; over 1000GFlops.

AMP aims at extreme scalability of single executables, from very simple hardware architectures of a single core processor with dedicated RAM, up to extreme scaling out in Cloud configurations. Sutter showed the aimed for heterogeneity in running an executable on a pc with a multi core CPU with onboard GPU, and also a double separate GPU installed.

My guess is that all this nice stuff also will reflect on the ways software can be build with the evolution of .Net and Xaml on Windows 8.

Pixel Shader Based Expansion

Best animation performance in Silverlight 4 is obtained from the combination of procedural animation and a pixel shader, as reported in a previous blog. A pixel shader is not really meant to be used for spatial manipulation, but in Silverlight 4 vertex and geometry shaders are not available. Also, pixel shaders are limited to Model 2 shaders in Silverlight 4, and only limited data exchange is possible. The question is then, “how far can we push pixel shaders model 2”. Another previous blog post discussed a preliminary version of dispersion. This post is about Expansion. The effect is much like Dispersion, but the implementation is quite different – better if I may say so. Expansion means that a surface, in this case a playing video is divided up in blocks. These blocks then move toward, and seemingly beyond the edges of the surface.

The Pixel Shader

The pixel shader was created using Shazzam. The first step is to reduce the image within its drawing surface, thus creating room for expansion. Expansion is in fact just translation of all the blocks in distinct direction that depend on the location of the block relative to the center of the reduced image.

In the pixel shader below, parameter S is for scaling, reducing, the pixel surface. Parameter N defines the number of blocks along an axis. So if N = 20, expansion concerns 400 blocks. In the demo App I’ve set the maximum for N to 32, which results in 1024 blocks tops. Parameter D defines the distance over which the reduced image is expanded. If the maximum value for D is the number of blocks, all the blocks (if N is even), or all but the center block (N is odd) just ‘move off’ the surface.

The code has been commented extensively, so should be self explanatory. The big picture is that the reduced, centered and then translated location of the blocks is calculated. Then we test whether a texel is in a bloc, not in an inter block gap. If the test is positive, we sample the unreduced, uncentered and untranslated image for a value to assign to the texel.

sampler2D input : register(s0);

// new HLSL shader

/// <summary>Reduces (S: Scales) the image</summary>
/// <minValue>0.01</minValue>
/// <maxValue>1.0</maxValue>
/// <defaultValue>0.3</defaultValue>
float S : register(C0);

/// <summary>Number of blocks along the X or Y axis</summary>
/// <minValue>1</minValue>
/// <maxValue>10</maxValue>
/// <defaultValue>5</defaultValue>
float N : register(C2);

/// <summary>Displaces (d) the coordinate of the reduced image along the X or Y axis</summary>
/// <minValue>0.0</minValue>
/// <maxValue>10.0</maxValue> // Max should be N
/// <defaultValue>0.2</defaultValue>
float D : register(C1);

float4 main(float2 uv : TEXCOORD) : COLOR
{
	/* Helpers */
	float4 Background = {0.937f, 0.937f, 0.937f, 1.0f};
	// Length of the side of a block
	float L = S / N;
	// Total displacement within a Period, an inter block gap
	float d = D / N;
	// Period
	float P = L + d;
	// Offset. d is subtracted because a Period also holds a d
	float o = (1.0f - S - D - d) / 2.0f;
	// Minimum coord value
	float Min = d + o;
	// Maximum coord value
	float Max = S + D + o;

	// First filter out the texels that will not sample anyway
	if (uv.x >= Min && uv.x <= Max && uv.y >= Min && uv.y <= Max)
	{
		// i is the index of the block a texel belongs to
		float2 i = floor( float2( (Max - uv.x ) / P , (Max - uv.y ) / P  ));

		// iM is a kind of macro, reduces calculations.
		float2 iM = Max - i * P;
		// if a texel is in a centered block,
		// as opposed to a centered gap
		if (uv.x >= (iM.x - L) && uv.x <= iM.x &&
		    uv.y >= (iM.y - L) && uv.y <= iM.y)
		{
			// sample the source, but first undo the translation
			return tex2D(input, (uv - o - d * (N -i)) / S );
		}
		else
			return Background;
	}
	else
		return Background;
}

Client Application

The above shader is used in an App that runs a video fragment, and can be explored at my App Shop. The application has controls for image size, number of blocks, and distance. The video used in the application is a fragment of “Big Buck Bunny”, an open source animated video, made using only open source software tools.

Animation

Each of the above controls can be animated independently. Animation is implemented using Storyboards for the slider control values. Hence you‘ll see them slide during animation. The App is configured to take advantage of GPU acceleration, where possible. It really needs that for smooth animation. Also the maximum frame rate has been raised to 1000.

Performance Statistics

The animations run at about 225 FPS on my pc. This requires significant effort as from the CPU –about 50% of the processor time. The required memory approaches 2.3Mb.

Pixel Shader Based Panning

Best animation performance in Silverlight 4 is obtained from the combination of procedural animation and a pixel shader, as reported in a previous blog. I know, a pixel shader is not really meant to be used for spatial manipulation. However, in Silverlight 4 vertex and geometry shaders are not available. Also, pixel shaders are limited to Model 2 shaders, and only limited data exchange is possible. The question is then, “how far can we push pixel shaders model 2”. Another previous blog post discussed a preliminary version of dispersion. This post is about Panning. Panning means here that the size and coordinates of a sub frame of, in this case a video, are changed.

The Pixel Shader

The pixel shader was created using Shazzam. The first step is to reduce the image within its drawing surface, thus creating a frame that works as a window through which parts of the video are visible. Panning is in fact just changing the coordinates of this window while correcting for the change in coordinates when sampling the source texture.

In the pixel shader below, the BlockSize in [0, 1], BlockX, and BlockY, both in [0, 1] define the panning window. The Bound function centers the panning window for coordinate values of 0.5. In the Main function, the ‘space’ variable denotes the available space to move the panning window around in. BlockH and BlockV denote the topmost horizontal and leftmost vertical edges of the panning window. We use these to filter input coordinates that sample the color texture. Other texels get assigned the background color of the demo application.

sampler2D input : register(s0);

// new HLSL shader

/// <summary>Size of BLock</summary>
/// <minValue>0.0</minValue>
/// <maxValue>1.0</maxValue>
/// <defaultValue>1.0</defaultValue>
float BlockSize : register(C2);

/// <summary>Horizontal Block selection</summary>
/// <minValue>0.0</minValue>
/// <maxValue>1.0</maxValue>
/// <defaultValue>0.5</defaultValue>
float BlockX : register (C3);

/// <summary>Vertical Block selection</summary>
/// <minValue>0.0</minValue>
/// <maxValue>1.0</maxValue>
/// <defaultValue>0.5</defaultValue>
float BlockY : register (C4);

float2 Bounds(float coord, float space)
{
	// Scale coordinate to available space
	float blockStart = coord * space;
	return float2(blockStart, blockStart + BlockSize); // 2nd argument is block end
}

float4 main(float2 uv : TEXCOORD) : COLOR
{
	/* Helpers */
	// Background Color
	float4 Background = {0.937f, 0.937f, 0.937f, 1.0f};
	// Available space to move around
	float space = 1.0f - BlockSize;

	/* Define Block */
	float2 BlockH = Bounds(BlockX, space);
	float2 BlockV = Bounds(BlockY, space);

	// If uv in BLock, sample
	if (uv.x >= BlockH.x && uv.x <= BlockH.y &&
	    uv.y >= BlockV.x && uv.y <= BlockV.y)
		return tex2D(input, uv);
	else
		return Background;
}

Client Application

The above shader is used in an App that runs a video fragment, and can be explored at my App Shop. The application has controls for panning window size, X-coordinate and Y-coordinate of the panning window on the video surface. The video used in the application is a fragment of “Big Buck Bunny”, an open source animated video, made using only open source software tools.

Animation

Each of the above controls can be animated independently. Animation is implemented using Storyboards for the slider control values. Hence you‘ll see them slide during animation. The App is configured to take advantage of GPU acceleration, where possible. It really needs that for smooth animation. Also the maximum frame rate has been raised to 1000.

Performance Statistics

The animations run at about 270 FPS on my pc. This requires both significant effort from both the GPU as from the CPU – both about 35% of the processor time. The required memory approaches 2.2Mb.