Introduction
In this tutorial we’ll be making an iOS app that renders the Mandelbrot Set and allows us to pan and zoom around it exposing beautiful and complex imagery.
The end result will look like this:
This tutorial we’ll be based on Fractals in Xcode 6 we’ll be using the shader code from there as a starting point.
Expand below to see the shader code.
void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
vec3 color = vec3(0.0,0.0,0.0); // initialize color to black
vec2 z = position; // z.x is the real component z.y is the imaginary component
// Rescale the position to the intervals [-2,1] [-1,1]
z *= vec2(3.0,2.0);
z -= vec2(2.0,1.0);
vec2 c = z;
float it = 0.0; // Keep track of what iteration we reached
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
if (dot(z,z) > 4.0) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
if (it < float(iterations)) {
color.x = sin(it / 3.0);
color.y = cos(it / 6.0);
color.z = cos(it / 12.0 + 3.14 / 4.0);
}
gl_FragColor = vec4(color,1.0);
}Download the Starter Project to follow along with this tutorial. You can find the final code at the end of the article.
Project setup
The GameScene.sks file contains a single sprite named fractal that fills the full scene with the shader Fractal.fshattached to it.
Fractal.fsh contains the shader code from above
GameViewController.swift contains code for setting up the game scene.
GameScene.swift is empty
If you run the code right now you’ll get the following result: 
Notice that the aspect ratio is fixed at 3/2. We’ll have to make it adjust based on the screen size.
Also, the image is static and you can’t interact with it in any way.
Setting up the view
We’ll be using a transparent scrollview for handling panning and zooming. The scrollview will automatically keep track of our position and our zoom level in the fractal.
Open the Main.storyboard file and drag a scrollview into the view. Make the scrollview fill the view and add constraints for width, height leading space and bottom space.
Set the scrollview’s max zoom level to 100000. We’ll be able to magnify the fractal up to one hundred thousand times! We can’t go any higher than that with this approach because we’ll reach the precision limit of float.
Drag a view into the scrollview. This view will be used for zooming the scrollview. The view itself will not display anything, we’ll use the contentOffset and zoom properties of the scrollView to update our shader. Make sure that the view fills the scrollView and you set constraints for width, height leading space, trailing space, top space and bottom space. Also set the view’s background color to Clear Color.
Next we’ll hook up the outlets we need and the scrollView’s delegate.
Drag outlets for the scrollView and the scrollView’s contentView.
class GameViewController: UIViewController, UIScrollViewDelegate {
@IBOutlet weak var contentView: UIView!
@IBOutlet weak var scrollView: UIScrollView!
...
}Next well stub out our delegate methods and implement the viewForZoomingInScrollView(scrollView: UIScrollView) -> UIView? method.
class GameViewController: UIViewController, UIScrollViewDelegate {
...
func scrollViewDidScroll(scrollView: UIScrollView) {
}
func scrollViewDidZoom(scrollView: UIScrollView) {
}
func viewForZoomingInScrollView(scrollView: UIScrollView) -> UIView? {
return contentView
}
...
}Sending data to the shader
A fragment shader can receive data from your Swift code via uniform variables. Uniform variables can be declared via the SpriteKit Editor. Let’s define some uniforms.
Open the GameScene.sks file and select the mandelbrot sprite. Scroll to the bottom of the inspector on the right and under “Custom Shader Uniforms” add 2 new items zoom of type float and value 1 and offset of type vec2. We’ll populate this uniforms with the scrollViews contentOffset and zoom properties.
WARNING: Xcode 6.3 has a bug with uniform variables. Uniforms won’t be initialized with the values you provide in the editor. You’ll have to initialize them from code.
We can access the shader on our node via the shader property. To get a uniform from the shader we use theuniformeNamed() method. For example to retrieve our zoom uniform we use:
let zoomUniform = node.shader!.uniformNamed("zoom")!Once we have a uniform we can change its value via one of of the properties
var textureValue: SKTexture!
var floatValue: Float
var floatVector2Value: GLKVector2
var floatVector3Value: GLKVector3
var floatVector4Value: GLKVector4
var floatMatrix2Value: GLKMatrix2
var floatMatrix3Value: GLKMatrix3
var floatMatrix4Value: GLKMatrix4We’re only interestead in using floatValue and floatVector2Value for this tutorial.
Ex: to set the zoom to 2 we use
zoomUniform.floatValue = 2Coordinate systems and mapping intervals
We’ll have to map between different coordinate systems while maintaining ratio. We’ll use this to convert the coordinates of our scrollview to the complex plane.
Lets first look at the 1D case:
To map a value x from the interval [0,a] to the interval [0,1] we just divide by the length of the interval x' = x / a.
To map a value x from the interval [0,1] to the interval [a,b] we multiply the value by the length of the interval and add the starting value of the interval to it x' = x * (b - a) + a
For example consider the x coordinate of an iPhone 4 in landscape. The x coordinate will be between 0 and 480. To map a value x to [0,1] we use x' = x / 480. To map x' from [0,1] to [-2,2] we use x'' = x' * 4 - 2
If we have a point with x coordinate 120 on our screen the corresponding point in the interval [0,1] will be 120 / 480 = 0.25 and in the interval [-2,2] it will be 0.25 * 4 - 2 = -1 as seen below.
Mapping between the scrollview and the complex plane
We’ll have to convert points on our scrollView to points in the complex plane. The first step is to convert points from the scrollView to points in the range [0,1]. To do this we need to divide the contentOffset by the contentSize. This will bring our contentOffset to the range [0,1].
var offset = scrollView.contentOffset
offset.x /= scrollView.contentSize.width
offset.y /= scrollView.contentSize.height
Our fragment shader provides points in the range [0,1] for both x and y coordinates which we’ll have to map to points that are in the interior of the scrollView’s contentView.
The normalized size of our contentView is 1.0 / zoom so the normalized coordinates of points in the contentView will be in the interval [contentOffset / contentSize,contentOffset / contentSize + 1.0 / zoom].
One more thing that we have to keep in mind is that the y-Axis points upwards in GLSL and the point (0,0) is in the bottom left corner. We’ll have to flip the y-Axis so that it matches our scrollView.
The following GLSL code converts the position to a point in the scrollView’s contentView:
// Fractal.fsh
void main {
vec2 position = v_tex_coord;
position.y = 1.0 - position.y; // flip y coordinate
vec2 z = offset + position / zoom;
...
}Below you can see in blue the frame of our scrollView’s contentView in both non-normalized and normalized coordinates.contentSize = (960,640) contentOffset = (240,160), and zoom = 2.0
ScrollView
Normalized ScrollView
Finally we have to map our point to the complex plane. To get a good look at the mandelbrot set we’ll want to map to the region [-1.5,0.5] x [-1,1] in the complex plane.
We’ll also want to correct for aspectRatio. Currently Both our x and y coordinate have the same scale. We’ll want to scale our x coordinate by the aspect ratio so that the image isn’t distorted.
Aspect ratio is the ratio of a screen’s width to its height.
// Fractal.fsh
void main {
...
z *= 2.0;
z -= vec2(1.5,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
...
}Below you can see our scrollView’s contentView mapped to the complex plane and corrected for aspect ratio.
To integrate all the above code we’ll create a new method updateShader that will pass the coordinates of our contentView to the shader. We’ll need to call the updateShader method in the scrollview’s delegate methods.
class GameViewController: UIViewController, UIScrollViewDelegate {
...
func updateShader(scrollView: UIScrollView) {
let zoomUniform = node.shader!.uniformNamed("zoom")!
let offsetUniform = node.shader!.uniformNamed("offset")!
var offset = scrollView.contentOffset
offset.x /= scrollView.contentSize.width
offset.y /= scrollView.contentSize.height
zoomUniform.floatValue = Float(scrollView.zoomScale)
offsetUniform.floatVector2Value = GLKVector2Make(Float(offset.x), Float(offset.y))
}
func scrollViewDidScroll(scrollView: UIScrollView) {
updateShader(scrollView)
}
func scrollViewDidZoom(scrollView: UIScrollView) {
updateShader(scrollView)
}
...
}Also don’t forget to call the updateShader method when the view appears so that you initialize the uniforms.
class ViewController {
...
override func viewDidAppear(animated: Bool) {
super.viewDidAppear(animated)
updateShader(scrollView)
}
...
}The shader code will finally look like this:
void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
position.y = 1.0 - position.y;
vec2 z = offset + position / zoom;
z *= 2.0;
z -= vec2(1.5,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
vec2 c = z;
float it = 0.0; // Keep track of what iteration we reached
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
if (dot(z,z) > 4.0) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
vec3 color = vec3(0.0,0.0,0.0); // initialize color to black
if (it < float(iterations)) {
color.x = sin(it / 3.0);
color.y = cos(it / 6.0);
color.z = cos(it / 12.0 + 3.14 / 4.0);
}
gl_FragColor = vec4(color,1.0);
}Challenges
1 . Optimization
The black areas of the set are the slowest to render. Luckily we can quickly determine if a point is in one of 2 large black regions (the cardioid or the region 2) as seen in the image below. Here you can find formulas for determining if a point is within one of these 2 regions. Improve the fragment shader by adding code that only performs the mandelbrot iterations if a point is outside these regions. This will greatly improve the apps performance when these regions are visible.
The main cardioid can be seen below in red and the region 2 in green.
Perform the mandelbrot iterations only if the point is outside one of these regions.
void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
position.y = 1.0 - position.y;
vec2 z = offset + position / zoom;
z *= 2.0;
z -= vec2(1.5,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
vec2 c = z;
bool skipPoint = false;
// cardioid checking
if ((z.x + 1.0) * (z.x + 1.0) + z.y * z.y < 0.0625) {
skipPoint = true;
}
// period 2 checking
float q = (z.x - 0.25) * (z.x - 0.25) + z.y * z.y;
if (q * (q + (z.x - 0.25)) < 0.25 * z.y * z.y) {
skipPoint = true;
}
float it = 0.0; // Keep track of what iteration we reached
if (!skipPoint) {
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
if (dot(z,z) > 4.0) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
}
vec3 color = vec3(0.0,0.0,0.0); // initialize color to black
if (it < float(iterations) && !skipPoint) {
color.x = sin(it / 3.0);
color.y = cos(it / 6.0);
color.z = cos(it / 12.0 + 3.14 / 4.0);
}
gl_FragColor = vec4(color,1.0);
}2 . Make a similar app that lets you explore the Julia set for some point c.
Ex: vec2 c = vec2(-0.76, 0.15);
void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
position.y = 1.0 - position.y;
vec2 z = offset + position / zoom;
z *= 2.0;
z -= vec2(1.0,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
vec2 c = vec2(-0.76, 0.15);
float it = 0.0; // Keep track of what iteration we reached
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
if (dot(z,z) > 4.0) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
vec3 color = vec3(0.0,0.0,0.0); // initialize color to black
if (it < float(iterations)) {
color.x = sin(it / 3.0);
color.y = cos(it / 6.0);
color.z = cos(it / 12.0 + 3.14 / 4.0);
}
gl_FragColor = vec4(color,1.0);
}3 . Add the point c as a uniform allow the user to adjust it’s value by panning around with 2 fingers
Use a UIPanGestureRecognizer to detect the 2 finger pan. You’ll have to normalize the translation that the gesture recognizer gives you
class GameViewController: UIViewController, UIScrollViewDelegate {
...
var c: GLKVector2 = GLKVector2Make(0, 0)
override func viewDidLoad() {
...
let panGr = UIPanGestureRecognizer(target: self, action: "didPan:")
panGr.minimumNumberOfTouches = 2
view.addGestureRecognizer(panGr)
}
func didPan(panGR: UIPanGestureRecognizer) {
var translation = panGR.translationInView(view)
translation.x /= view.frame.size.width
translation.y /= view.frame.size.height
c = GLKVector2Make(Float(translation.x) + c.x, Float(translation.y) + c.y)
let cUniform = node.shader!.uniformNamed("c")!
cUniform.floatVector2Value = c
panGR.setTranslation(CGPointZero, inView: view)
}
}void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
position.y = 1.0 - position.y;
vec2 z = offset + position / zoom;
z *= 2.0;
z -= vec2(1.0,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
float it = 0.0; // Keep track of what iteration we reached
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
if (dot(z,z) > 4.0) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
vec3 color = vec3(0.0,0.0,0.0); // initialize color to black
if (it < float(iterations)) {
color.x = sin(it / 3.0);
color.y = cos(it / 6.0);
color.z = cos(it / 12.0 + 3.14 / 4.0);
}
gl_FragColor = vec4(color,1.0);
}4 . Use an image to color a julia style fractal. There are many ways to accomplish this, an interesting one is using the following approach:
- At each iteration get the color from the image coresponding to
z. If the color is not transparent break out of the loop. - If the obtained color is non transparent after running all the iterations use it to color the coresponding pixel
- If the obtained color is transparent use a different formula to color the point. for example the normalized number of iterations.
Here’s a julia set colored with the image of a bunny.
You’ll have to add another uniform of type Texture named image. To get a color from the texture at position p you can use vec4 color = texture2D(image,p);
class GameViewController: UIViewController, UIScrollViewDelegate {
...
override func viewDidLoad() {
...
let imageUniform = node.shader!.uniformNamed("image")!
imageUniform.textureValue = SKTexture(imageNamed: "bunny")
}
...
}vec4 getColor(vec2 p) {
if (p.x > 0.99 || p.y > 0.99 || p.x < 0.01 || p.y < 0.01) {
return vec4(0.0);
}
return texture2D(image,p);
}
void main() {
#define iterations 128
vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1]
position.y = 1.0 - position.y;
vec2 z = offset + position / zoom;
z *= 2.0;
z -= vec2(1.0,1.0);
float aspectRatio = u_sprite_size.x / u_sprite_size.y;
z.x *= aspectRatio;
vec2 c = vec2(-0.76, 0.15);
vec4 color = vec4(0.0); // initialize color to black
float it = 0.0; // Keep track of what iteration we reached
for (int i = 0;i < iterations; ++i) {
// zn = zn-1 ^ 2 + c
// (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
color = getColor(z);
if (dot(z,z) > 4.0 || color.w > 0.1) { // dot(z,z) == length(z) ^ 2 only faster to compute
break;
}
it += 1.0;
}
if (color.w < 0.1) {
float s = it / 80.0;
color = vec4(s,s,s,1.0);
}
gl_FragColor = color;
}5 . Experiment with formulas for Mandelbrot like fractals This is a open ended challenge. 2 examples are provided below
Burning Ship Fractal
Formula zn = abs(zn-12 + c)
GLSL
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += c;
z = abs(z);Sierpinski Julia
Formula zn = zn-12 + 0.5 * c / (zn-12)
GLSLS
vec2 powc(vec2 z,float p) {
vec2 polar = vec2(length(z),atan(z.y,z.x));
polar.x = pow(polar.x,p);
polar.y *= p;
return vec2(polar.x * cos(polar.y),polar.x * sin(polar.y));
}
void main() {
...
z = vec2(z.x * z.x - z.y * z.y, 2.0 * z.x * z.y);
z += 0.5 * c * powc(z,-2.0);
...
}
Hi Silviu Pop,
Nice article. Do you have any idea on how to display charts (Visualizations like D3) using swift ?
Thanks.
Hi,
Have a look at iOS Charts and PNChart which has a swift version.