KnitML

Hey! What's news?

I haven't posted since April, so you may be wondering what happened to me (and consequently the project). Well, after the last release, I decided to take a much-needed break and wait for some new people to take interest. That came in the form of i0lanthe last month.

I know that there are a lot of people out there that want to help out but don't really know how to get started. Well, the Ravelry KnitML Forum has been abuzz in recent weeks, and the first-cut of a first draft of a KEL tutorial has emerged. I believe that this tutorial is a sorely needed stepping stone to boost involvement of the knitting community. Try it out and give us your feedback.

In other news, now that both project source code repositories are being hosted at Google Code, I may try to move the bug tracker over there. It also appears that there is a Mylyn Connector for it (which, if it works, is the only thing that would be stopping me from migrating from Bugzilla.) I'll let you know.

Using KnitML, Part Two: Customizing the Result

Previously in this series, I showed you how I converted Jenna Wilson's Banff sweater to a KnitML pattern. In this post, I'm going to show you how you can customize the pattern to your preferences.

For those of you reading this post who are not involved with the KnitML Project, I'm asking that you keep an open mind and not judge or dismiss this software based on its current limitations. Please remember that this is beta software, supported and nurtured by the knitting community. Your suggestions for improvement are most welcome (and, in fact, sought after). The software needs to be made better, and only you can make this possible. You can leave a comment on the blog, you can contact me privately (by clicking on the Contact Me link on the sidebar). or you can post a thread on the Ravelry KnitML group. Thanks!

When I open up the KEL editor, notice that there are three different views of the KnitML pattern: Code, XML, and Pattern.

The Code View is where I get to work with and edit the pattern. The XML view shows the pattern in true KnitML format, derived directly from the Code View. If I were to publish this KnitML pattern online, this is the file I would share. The Pattern View shows how the pattern looks using my personal preferences.

The pattern editor ships with a set of sensible defaults for the pattern view. Here is what Banff looks like out-of-the-box:

While this is certainly fine, it's not entirely to my liking. I can open up the Preferences page and make a few changes.

I'm not a huge fan of Arial. Garamond would look much nicer.

To get the pattern to re-render, I must navigate away from the Pattern View (either to the Code or XML View) and then back to the Pattern View. (There will be an easier way to do this in a future release.)

Notice the gauge is expressed as a 4-inch swatch in stockinette stitch.

You may recall that I expressed the gauge like this:

Gauge 'stockinette' {
      StitchGauge: 3.5 stitchesPerInch
      RowGauge: 5.25 rowsPerInch
    }

The Pattern View does the conversion to a square for me. If you don't care for gauges written this way, you can uncheck the "Describe gauge as a square" option in the Preferences page.

We intend to support non-stockinette-based gauges in the future.

If you are familiar with the original pattern from Knitty, you probably know that it did not come with charts. You also may remember that I did not type in anything that looked like a chart when I created the pattern in the Code View.

I can tell the renderer to construct charts from the KnitML pattern. To do this, go to the Preferences page, expand the Rendering option on the left, click on Charts and then check the box labelled "Enable chart creation for instructions." The renderer will make a chart whenever it can.

Here is a part of the pattern that shows charts well:

(The legend has some cleanup work slated for the next release.)

By default, the renderer uses simple text art for its symbols (based on the Unicode character set). Since I have the Aire River Knitting Font installed on my machine, I could use those symbols instead.

If I wanted to manipulate the chart further, I could copy and paste it directly into a spreadsheet.

Note that if you don't use Windows and Excel, it may take a little more work to get it into this form. While the copy/paste with Firefox and OpenOffice does work, the font settings are not preserved. In order to get the Aire River font to look correct, you would have to re-apply that font to the cells of the chart.

Notice that the charts are all right justified with respect to their "no stitch" elements. In a future release, the renderer will be able to infer whether to center, left, or right justify depending on conditions. This will help keep the chart looking more like the finished object. For more complex charts, however, auto-justification may not always be possible. A designer may choose to specify the placement of every "no stitch" element themselves to guarantee their own chart layout.

So how do you write patterns in KnitML which were originally designed as a chart? Right now, the process to convert a chart to KnitML is manual (by writing out the appropriate KEL syntax). This is undeniably awkward. In the future, existing software (such as but not limited to Knit Visualizer) could make this a whole lot easier by creating the KnitML chart instead of you having to do it.

I can also customize the instructions themselves. Here are the defaults for several common decreases:

Suppose I knit in combined style (or left-handed) and prefer all decreases to indicate their lean. I can edit my preferences to display RLD or LLD (right-leaning decrease, left-leaning decrease):

That will end up looking like this:

Suppose French is my native language. I can have the pattern rendered in French using international units:

It will render all of the non-custom instructions in French automatically.

A pattern designer can provide translation files for their customized sections. The Nutkin2 sample demonstrates how to fully internationalize a pattern. Future releases may be able to provide an "auto translate" feature if no translation is found for the given language.

I've touched on the major points of the Pattern Editor. If you have any questions, ideas, or thoughts, please feel free to comment here on the blog, contact me privately, or ask a question on the KnitML Ravelry group or the public Answers forum.

Pattern Editor 0.2.0 and Core Tools 0.5.0 released

I am very pleased to announce the arrival of both the KnitML Pattern Editor 0.2.0 and KnitML Core Tools 0.5.0. This is the first truly usable version of the Pattern Editor.

Features of this release include:

A great new look (thanks, Susan!)
A great new name (thanks, Ravelry!)
A new sample: the Banff sweater (thanks, Jenna!)
A new help system
Customizable rendering preferences
Charting support
Stitch holder support
Knitting Expression Language improvements
French rendering (thanks, Kajin!)

Get yourself a copy here, and feel free to bring any of the samples into the editor to try out the tool.

Thanks to everyone who made contributions!

Very, very close to 0.5.0

At this point, both KnitML Core Tools and, more importantly, the KnitML Pattern Editor (formerly GPEC) are in Release Candidate 1 status. Visit the milestones links on the sidebar to download. (Sorry, the Pattern Editor at this time is only available on Windows. I don't expect it will be in RC1 status very long.)

Looking Forward: A Request for Help

With KnitML Core Tools 0.5.0 coming out in the next few weeks along with the new Editor 0.2.0, it's a good time to stop and reflect about the project and where we want to go next.

I've learned quite a bit about myself over the years I've worked on KnitML. (Yikes, has it really been years? It will be 2 years in September.) Anyways, there are some things I'm good at, and there are some things I'm just not all that great at (lousy, in fact). Sparing you my personality quirks, here is what I've come up with.

I feel that I'm pretty good at:

Designing software for flexibility and change
Thinking through scenarios
Project Scoping

I've learned that I am inexperienced in the following areas:

Graphic Design
Pattern Design
Documentation
Internationalization (i.e. dealing with multilingual issues)

Susan has helped improve the editor far past what I would have been able to do myself. While it's still very much a work-in-progress, version 0.2.0 of the editor is not only starting to pass for decently configurable software, but it's also starting to look and feel really nice. If you are a graphic designer (or you just have strong opinions about how your UI feels), please give us your thoughts!

Janet has brought the pattern designer's perspective to the project, and it's been extremely valuable (not to mention eye-opening). If you are a designer and have thoughts about the project, please make your voice heard!

As for documentation, I can certainly write manuals in a way that I can understand them. This is, in fact, a pretty useless skill. :) Certainly the reference table for Knitting EL is comprehensive, but it's not clear to me that it's actually usable. I get the feeling that people want to learn about KnitML, but they open up the User's Guide, read a bit, and then just give up.

If you feel you have good writing and documentation skills, and if you are curious about KnitML but have been unsure about where to start, this is your chance! I'm looking for people to dive in feet first, ask any and all questions when they get stuck, and keep track of how they are approaching KnitML. Click here to send me a note.

Multilingual support is also an area I need help with. While the approach I've been taking works, it's pretty clear I need a better system for long-term maintenance of language files. If you are a professional translator, I need to hear your perspective about translating documents and software. I've been trying to glean some of these things by doing online research, but I really am not an expert here. Any advice at all would be most welcome.

And, of course, I'm always looking for programmers to lend a hand. Some previous Java experience is very helpful, but if you have skills in other technologies that you think can bring value to the KnitML Project, please share your ideas!

Got other skills another area you can help out with? Just let me know.

KnitML Core Tools 0.5.0 RC1 available

All issues slated for the 0.5.0 release have been resolved. This includes adding stitch holders, charting support in three fonts (text art, Knitters Symbols, and Knitting Font 2.00), HTML patterns, and much more. The XML schemas have been finalized for the 0.5.x series and have been uploaded to the web site.

The graphical editor (formerly GPEC, soon to be called something else) will be released as 0.2.0 simultaneously with 0.5.0 final.

You can download 0.5.0 RC1 from the milestone releases page.

Last Call for Bug Submissions for the 0.5.0 Release

If you want your opinions put into an actionable plan, submit an issue today. This is the last call before the list is frozen for the 0.5.0 relesae (and 0.2.0 of the GPEC release, which will include 0.5.0).

XText and KnitML

So I'm here at EclipseCon 2009. Someone tipped me off about XText a few months back, so I thought I would go to the talk.

It sounds like it has a lot to bring to the table for the Knitting Expression Language. It has a way to express the KEL grammar in a formal way, which means that it is also capable of providing content assist / code completion, syntax highlighting, and pattern formatting for the KEL format.

XText would also enable the KEL format to take the form of an external DSL. As Martin points out, external DSLs traditionally take a lot of work. But XText looks like it could make this a lot simpler. At the basic level, all I need to do is write the grammar, and that gives me a base working set of tools as a starting point.

I could also get rid of many of the curly braces. This, for instance:

Directions {
  InstructionGroup 'make-back' withLabel "Make the Back" {
    ...
  }
  InstructionGroup 'make-front' withLabel "Make the Front" {
    ...
  }
}

could look like this:

Directions:
  InstructionGroup "Make the Back":
    ...
  InstructionGroup "Make the Front":
    ...

Because the structure of the grammar at this level of the pattern is very formal, you don't really need open / closed curly braces. You can simply infer where one InstructionGroup starts by where the next one begins. This looks much more like a knitting pattern to me. But the current way the internal DSL is built (text polishing on top of Groovy), you really need the curly braces so that Groovy understands where these boundaries are. It sounds like XText could liberate us from unnecessary braces.

XText will also be able to provide validation and feedback to the user about their KEL pattern. That's going to be extremely important, because currently the feedback that the conversion tool provides to the user is a stack trace (with a fairly cryptic error message).

All of this hits home a point I have made before. If you are writing software to integrate with KnitML, please make sure you're coding to the XML specification and not the KEL syntax. The KEL syntax is really a tool to achieve an easy way to create a XML document. I would consider the KEL not stable for integration between versions. At least not at this point.

Using KnitML, Part One: Creating the Pattern

With the upcoming release of KnitML Core Tools 0.5.0 (and the editor that goes with it), I wanted to show you how I approached converting an existing pattern, the Banff sweater designed by Jenna Wilson, to a KnitML pattern.

First, I created a project named Banff.

Then I created the KEL file...

...renaming the default file name to banff.kel.

Here is what I started with. This is a template that highlights the basic structure of the document.

A KnitML pattern consists of exactly one Pattern element. The Pattern element itself has a GeneralInformation, a Supplies, and a Directions element, and those in turn have elements. If an element cannot be defined all on one line, its contents are indicated by curly braces.

Then I took a look at the original pattern and put that information manually into this document. Since the pattern is given in two sizes, I picked the S-M size to convert.

Elements like Name, Description, and Dimensions were fairly self-explanatory. I converted the square gauge (14 sts x 21 rows = 4 in) into decimal form (3.5 stitchesPerInch, 5.25 rowsPerInch). Knitters who like to read square gauge have an option to create their pattern that way, so I didn't need to worry about that.

For sake of clarity and convention, items surrounded by double quotes are different from items with single quotes. Double quotes refer to literal text or a label. That is, they are written in a particular language (English in this case, as specified by the "en" value by the Pattern element). The values for Name and Description, for instance, are surrounded by double quotes.

Single quotes refer to system IDs. These IDs provide a way to refer to the element upon where they appear in a different part of the pattern. It's a good idea to make them readable, but they don't have to be (they could be generated, for instance). IDs have a few restrictions. They must be unique in the pattern, and they cannot start with a number or cannot contain spaces. (They follow the NCName rules for XML IDs, if that means anything to you.) Additionally, an element cannot refer to an ID that is defined later in the document (elements are processed from the start of the file to the end). That is why they appear with dashes where spaces would normally be ('yarn-type1' or 'needle1', not 'yarn type 1' or '1st-needle').

The original pattern has a section called Patterns. It defines three sets of instructions for use in the pattern directions. The 2x2 Rib pattern is a two-row instruction. RS Decrease Row and WS Decrease Row never appear apart, so I consolidated them into an instruction called Decrease Rows. Also, since the pattern needs to refer to Stockinette Stitch, I included that definition as well. To make these three instructions appear in the header, they went into the Directives and InstructionDefinitions elements, which appears directly under the Pattern element.

Notice that Row elements do not have to have row numbers; they are optional. When they appear in the header like this, if they are not provided, it is assumed that each instruction starts with row 1. (When they are in the body, row numbers will start with row 1 and continue upwards until they are reset.) Additionally, instructions in the header must specify how they are to be worked: flat or round.

Next come the directions themselves. The original pattern is organized into major sections: Back, Right Shoulder, Left Shoulder, Front, Left Shoulder, Right Shoulder, Sleeves (2), and Finishing. In KnitML, each of these sections correspond to an InstructionGroup element. Each InstructionGroup gets an ID (to refer to) and a label (to display).

An InstructionGroup element can be made up of Section elements. Each Section element corresponds to a paragraph in the original pattern. I did my best to make these correspond to the way the pattern was organized.

Here is the InstructionGroup for the back:

I consolidated a few things in this InstructionGroup. The original pattern has the 2x2 ribbing repeat listed as a 42-row repeat. To me, it seemed like the designer meant the ribbing to be repeated for 8 inches, since the row gauge was specified at 21 rows to the inch. I chose to express it by length because it seemed a bit more natural (and possibly more accurate in case a knitter was not getting the exact row gauge). I interpreted increasing 4 stitches "more or less evenly" across 86 stitches to mean k21, m1, k22, m1, k21, m1, k22, m1. (In the future, there may be a way to express an even increase like this, but right now the designer has to be explicit.)

An InformationalMessage element provides a way for the designer to add helpful advice to the pattern. This can be anything from a detailed explanation of a technique to informal suggestions about altering the pattern. Additionally, there will be a way for an InformationalMessage to provide a "summary" explanation of more detailed knitting. In this case, the designer will provide the summary and then the details of the technique, and the knitter can choose to "fold up" the details if they don't want to see them (or the pattern can start "folded up" if they prefer it that way).

Notice how some parts of the pattern have a "State sts" element, while others say "State 34 sts." I did this mostly to show both ways of doing this. "State sts" will simply state the number of stitches currently being worked (whatever that number is). "State 34 sts" will do the same, but it will ensure that there are 34 stitches being worked and throw an error if it's not. This can help identify copy and paste errors.

The next section is the back right shoulder.

This section highlighted some deficiencies in the current KnitML specification. For instance, there is not yet a way to express putting stitches on a holder in KnitML. (This will change in an upcoming version of the specification.) As a result, there is an abnormal bind off / cast on combination in this InstructionGroup that doesn't quite follow the original pattern. Please forgive this section for now. :)

I then went on to transcribe the Left Shoulder (Back), Front, Left Shoulder (Front), Right Shoulder, and Sleeves. The Sleeves required a level of detail which was merely summarized in the original pattern. Since KnitML needs to know exactly how to work the stitches, I had to derive the details based on the general instruction. (This will be implemented as a foldable instruction in the future.)

Here is how I generally approached transcribing each row from the original pattern. I started out copying each row from the web site and pasting in into the KEL file. Then I did some minimal altering to make it fit into the KEL format. For example, the original pattern had a purl 1 through back loop expressed as 'p1tbl.' In KEL this is actually 'p1 tbl' (with a space). [RS] / [WS] also look like 'rightSide' and 'wrongSide' instead. Also, the desinger used spaces to delineate her row labels, whereas KEL expects colons. (At some point, some of this expected formatting may be able to be customized by the designer. We're not there yet, though.)

Each row takes the form of:

Row [number(s)] ("rightSide" | "wrongSide"): operation(s)

Row [number(s)] ("rightSide" | "wrongSide") {
  operation
  operation
}

For instance:

Row 1: k to end
Row 2 wrongSide: p to end
Row [3,5,7]: k to 2 before end, p2
Row [4,6,8] wrongSide {
  p to 2 before end
  k2
}

If you don't want the row number to appear, use the NextRow element instead. If you have a short row, use ShortRow.

If you specify multiple row numbers that are not sequential (such as Row [4,6,8] above), you must surround the complete set of rows with an Instruction element. So the above rows would actually look like this:

Instruction 'inst1' {
  Row 1: k to end
  Row 2 wrongSide: p to end
  Row [3,5,7]: k to 2 before end, p2
  Row [4,6,8] wrongSide {
    p to 2 before end
    k2
  }
}

The last InstructionGroup was the finishing instructions. The KnitML Specification currently doesn't have a good way to express picking up along an edge, so this section is done as InformationalMessages surrounded by Sections. Defining edges will be in a future spec.

That is how I created the pattern. In the next post, I'll show you what I'm able to do with the pattern once it's in this format.

Why an Expression Language is Important to Get Right

I heard Martin Fowler give a talk a few weeks back about domain-specific languages (DSLs) and their role in software development of the future. He validated a lot of the assumptions I have been making along the way with the Knitting Expression Language (a DSL for knitting patterns), so it certainly feels good to know that I'm on the right track.

One of the most interesting points he made about DSLs was that they can enable people who don't think of themselves as programmers to program. His example was Cascading Style Sheets (CSS). Web designers don't usually consider themselves to be programmers, but they are usually proficient in writing style sheets. Since CSS is a language that is targeted for a very specific purpose (describing presentation of HTML pages), it doesn't "feel" like a programming language. Yet that's what it is. It's not a turing-complete language, but that's one of its strengths.

DSLs, in fact, aren't limited to computer languages; they can be relevant to language in general. Martin's example of this was the jargon required to order your favorite, multi-adjective drink from Starbucks. A Grande, Non-Fat, Vanilla, Decaf Frappuccino with Whip doesn't have much meaning outside of Starbucks, yet it is a concise way to communicate your specific drink preference to the barista. (By the way, this is not my favorite Starbucks
drink!)

It should come as no surprise, then, that directions for any knitting pattern use terminology from an informal DSL. There are relatively standard abbreviations for knit, purl, slip, etc. Even though a legend is usually provided with the pattern, most knitters do not need to refer to it to understand that, for instance, k2tog
means "knit the next two stitches together through their leading loops, creating one stitch out of two." We have to keep this in mind when creating a computer-based DSL for knitting patterns. Since there is already a DSL for expressing patterns, we should make our DSL look as much like the existing one as possible.

I've run across several knitting applications that claim to have "pattern-like" syntax, but in reality its expression is significantly hampered by the underlying programming language. This from the
Knitter project, for instance, still feels very much like a programming language:


sk fix(-3,0,0) co 7 fix(3,0,0)
turn k 8
turn k 2 k2tog k 2
turn toEndk

It would be fairly unreasonable to expect designers to express their patterns in this format.

This knitting chart generator has much clearer syntax, yet the square brackets around many instructions make this a bit awkward:


[k]8
[k]2,[k2tog]2,[k]2
k,k,k,k,k,k,k,k

The tool makes this somewhat easier to deal with by providing a graphics for create the chart. However, point-and-click solutions are traditionally not very fast. What if I were transcribing row-by-row instructions out of a book? In that case I can use type in the syntax, but now I'm hampered by the square brackets. What if the pattern were published on a web site? I would want to copy/paste the directions, but short of coming up with a fancy regular expression, I would have to do this by hand.

Now, I'm not meaning to knock the work of the Knitter or Orangellous folks. They've done a lot of work to create their software, and I'm sure their programs will provide (or already have provided) value to the knitting community. And, to be fair, the usability of their syntax was not top priority. However, they may want to consider that an awkward syntax has the potential to get in the way of adoption.

You may be wondering how close you can get to the look and feel of a real pattern while still writing something that a computer can understand. If you've been reading this blog, you can probably guess. :)

You could write the above directions like this in the KnitML Expression Language (KEL):

co 8 sts
Row 1: k8
Row 2: k2, k2tog twice, k2
Row 3: k to end

This is indistinguishable from a real pattern. Yes, there is a bit of syntactic baggage to set up the pattern, and I need to take a closer look at improving some of it. In the interest of full disclosure, the full pattern would look like this:


Pattern {
  Directions {
    co 8 sts
    Row 1: k8
    Row 2: k2, k2tog twice, k2
    Row 3: k to end
  }
}

It's also worth nothing that KEL patterns are not KnitML patterns, per se. Rather, they translate into KnitML patterns. KnitML is an application of XML, which represents a knitting pattern's canonical form. A canonical form is very important as an interchange format between software components, because the software processing is greatly simplified. Having a DSL sit on top of the canonical form gives us the best of both worlds. We get pattern readability up front, while programmers don't have to write software for every conceivable expression difference.

To put it another way, KEL allows you to write patterns in a familiar way. In turn, it translates that into an internal standard that software of all kinds will rally around. Then any knitter can render the KnitML pattern in a form that's most familiar to the way they read patterns. Computer DSLs don't have to be syntactically perfect for them to be usable, but they should be clear enough to be meaningful and easy to use. The more meaning their syntax conveys, the less intimidating they will seem to the average user. Programmers can keep "computery" languages to themselves.