Let's teach a neural network to trace bitmaps

Flesh out your ideas for new or improved Inkscape features before submitting a request.
unfa
Posts: 1
Joined: Wed Mar 15, 2017 1:16 am

Let's teach a neural network to trace bitmaps

Postby unfa » Wed Mar 15, 2017 10:11 pm

If you have too much time on your hands and love developing cool stuff - this is probably for you :D

I've recently saw some examples of what deep neural networks can do. I wonder if they could be used to "intelligently" trace bitmaps creating vector shapes form low-res images?

Possibly preporcessing the image with a sobel & canny edge detector would be useful.

Probably such a neural network could "see" a fixed part of the input image (a circle of for example 8 pixels radius) and travel along the shape's edge, creating/refining bezier curve nodes and control points as it goes around.

Possibly useful resources:

https://www.youtube.com/watch?v=S9Y_I9vY8Qw - a neural network learning to play a racing game
https://www.youtube.com/watch?v=uihBwtPIBxM - finding edges in a bitmap: Sobel
https://www.youtube.com/watch?v=sRFM5IEqR2w - finding edges in a bitmap: Canny edge Detector
http://machinelearningmastery.com/tutor ... hon-keras/ - tutorial in creating a simple neural network in Python using Keras library

I guess training /evaluation data could consist of pairs of svg files of different black shapes on white background, and bitmap exports of these shapes.

The net would try to create a bezier path as close to the original one, and then a high resolution rendering of both (approximated and original beziers) can be pixel compared to define the score. Each different pixel increases the loss.

Example:

We draw 1000 random convex black shapes on white background in a 256 x 256 canvas.
All of these get rendered 1:1 and converted to grayscale PNG.
The neural net tries to approximate the beziers based on that low-res bitmaps.

Then we render the neural-net created beziers with 4x the resolution (1024x1024) and compare that to the 1024x1024 rendering of the original shapes.
Each pixel that isn't identical on both images increases the loss, so we can asses how well it did.

Tadam!

There are some holes to fill in here, but this at least can be fun. At most - it could become really useful one day.


User avatar
brynn
Posts: 9104
Joined: Wed Sep 26, 2007 4:34 pm
Location: western USA
Contact:

Re: Let's teach a neural network to trace bitmaps

Postby brynn » Thu Mar 16, 2017 8:58 am

For those of us who are not very technically inclined or aware - what is the benefit of tracing by a neural network and tracing by Trace Bitmap?

v1nce
Posts: 610
Joined: Wed Jan 13, 2010 4:36 am

Re: Let's teach a neural network to trace bitmaps

Postby v1nce » Fri Mar 17, 2017 4:37 am

with NN:
10x better
1000x slower
:D

tylerdurden
Posts: 1481
Joined: Sun Apr 14, 2013 12:04 pm
Location: Michigan, USA

Re: Let's teach a neural network to trace bitmaps

Postby tylerdurden » Fri Mar 17, 2017 5:23 am

:lol: :lol:

I gotta fast machine though, so it might be worth it.... Think how long cleaning up traces takes.

The code is young too, so maybe some streamlining will help.
Have a nice day.

I'm using 0.92 64 bit win8.1


The tutorials in the Inkscape Help menu are fun and the best place to start.

Most other answers are in the Inkscape Manual! :-)
http://tavmjong.free.fr/INKSCAPE/MANUAL/html/index.html

User avatar
brynn
Posts: 9104
Joined: Wed Sep 26, 2007 4:34 pm
Location: western USA
Contact:

Re: Let's teach a neural network to trace bitmaps

Postby brynn » Fri Mar 17, 2017 5:28 am

1000x slower? Would that be worth the effort? Or maybe we are talking about a cool toy, which the fun trumps the bumps?


Return to “Inkscape Ideas”

Who is online

Users browsing this forum: No registered users and 1 guest