Forum Samples, Tips and Tricks

Markup Creator – Open Source Project

floba
Hi guys,

I am working on an editor for markup, designed to create paged documents especially with XML and CSS using Prince XML. Take a look on http://www.markup-creator.org/ and the feature video.

The aim is to have preferably a better cross platform IDE for writing WYSIWYM documents like TeXnicCenter (but cross platform! and not only for LaTeX, but especially for XML and CSS). But Markup Creator is absolutely not completed yet and very buggy. That's why I am writing to this forum. Maybe there is somebody out there who wants to contribute – that would be great! E.g. in the feature video there is a script, that automatically creates a table of contents, list of figures and list of tables that prince can't do. Prince also can only render math written in MathML. But MathML is nothing for humans, TeX-math is much easier to write. Do it inside the XML document and this script converts it to MathML using blahtex! But the script is bad formed and not platform independent. It also should make it possible to use e.g. HAML instead of XML and SASS, SCSS or LESS instead of CSS (see http://www.markup-creator.org/resources/files/Ml+Style2Pdf.pdf). So if some of you is familiar with e.g. Python or Ruby (or another script language?), do you fancy? The editor itself also is hardly usable at this point of time. So if some of you are familiar with C++ and Qt...

Looking forward to hearing from some of you :-)

Best regards,
floba
mikeday
Prince 8.0 will support JavaScript and the DOM, which may be an easier way to write scripts that can generate tables of contents or convert LaTeX style equations into MathML.
floba
I am impatiently waiting for a new release, especially because of better math support (http://www.princexml.com/bb/viewtopic.php?f=4&t=3714&start=0&hilit=math). Do you already have a release date?

As far as I know, you cannot call command line tools using JavaScript. So it will not be possible to convert TeX-math to MathML using blahtex or convert SASS to CSS with sass.
mikeday
The 8.0 beta release will be online in the next day or so. It is true that in Prince, JavaScript cannot call command-line programs, but there are solutions like jsMath for converting TeX mathematics to MathML.
jim_albright
Nice product presentation.

Jim Albright
Wycliffe Bible Translators

floba
The announced script has been written: With Ml+Style2Pdf it is also possible to render HAML, SCSS, SASS and/or LESSCSS – all without a project file or anything else: just call
$ ./Ml+Style2Pdf.py [file]

Ml+Style2Pdf parses this "main file" recursively and renders the files as expected.

Even more powerful, there are "magic elements" to generate a table of contents or a list of figures automatically with <toc> and CSS selectors or an index with <idx> and CSS selectors. Examples:
  • Table of contents:
    <toc sel1="div#chapter h1" sel2="div#chapter h2" sel3="div#chapter h3"/>
  • List of figures:
    <toc sel1="div#chapter div.figure p"/>
  • Index:
    <idx sel="span.idx"/>
Inside the elements, an ul li a-list is generated.

The <math>-element is extended, too: Now it is possible to write TEX- and ASCII-math instead of MathMl:
  • TeX-Math:
    <m:math parse="tex">
        \mathrm{e}^{-i \pi} + \frac{3}{2} = \frac{1}{2}
    </m:math>
    
  • ASCII-Math:
    <m:math parse="ascii">
        e^{-i pi} + 3 / 2 = 1 / 2
    </m:math>
    

The equations will be converted to MathML. After all those conversions, prince is called finally and all temperary files are deleted.

Take a look on http://www.markup-creator.org/resources/files/Ml+Style2Pdf.pdf for a complete example and more information. You can download the sources on http://www.markup-creator.org/#Scripts.

The script is tested on Mac OSX 10.6 and Linux Ubuntu 10 successfully. I still suspect some bugs, so I would be glad if you report them on https://bitbucket.org/floba/markup-creator/issues.
Nicholas
It looks like a good start, but I have to ask, why should anyone want a markup editor? Sorry to put a spanner in your spokes, but it has to be pointed out that it makes no sense assuming that the disk interface, the file format, is the same as the user interface. In fact, it's the opposite. Editing markup by hand is a terrible relic of the old days, before we had worked out how to do user interfaces. Perhaps you decide that the best way to serialise and store a document is to put little <h1> tags around strings, or \chapter{}. Text-based formats are handy to debug and work with sometimes, but once you've made a design decision about storage, it's mad to force the user to pretend to be a computer. Why should it be that \-c-h-a-p-t-e-r-{ is the best way to represent headings to the user? Markup editing rightly ought to be gone, because besides its use for debugging, it's a total dead-end in terms of UI. Have you checked out software like XXE? It's too technical for a lot of users, and that sort of interface would need some heavy tweaking if you wanted it to challenge a Word-style UI, but it's a big improvement conceptually. We need to get out of the mindset that conflates file format with UI.

Basically, if a markup editor forces you to type the <h1> tag characters, it is far too slow, and unusable. If it is even close to the speed of XXE or Oxygen's shortcuts, then the actual keyboard input is going to have to be much more concise than the markup, in which case, what thinking could justify ever showing it to the user? Remember, to get properly fast and usable, a markup editor is going to have to implement some kind of relaxng framework to do validation guided markup, to make it quick to use by picking the next elements to insert.

I currently use XXE a lot, and the DocBook style sheets handle ToC/Index generation much better than anything I would have time to write myself. Take a good look at XSLT first before doing too much JS/procedural rearranging of your documents. (Of course, you use an IDE rather than type the XSLT tags yourself!) Don't knock the XSLT rule-based framework: when you do things right, it is the king of XML-like processing stages at the moment (see the mess the Drupal crew are in for instance http://developmentseed.org/blog/2009/mar/01/limitations-drupal-theme-layer).

So, the way I see it is that a markup editor really needs to justify itself now. Takeaway: think first about what the best abstraction for a document is and how the user should interact with that before assuming that users and programmers acting as users should be thinking like programmers. Sorry to question your project, which looks like it is very good at what it sets out to doSomething instead I would be interested in, and haven't yet seen, is more clever version of XXE which does a bit less, but is written in browser.
floba
why should anyone want a markup editor? […]

This is a discussion on philosophy: If you like to do WYSIWYG, MS Word, Oo.org Writer, Adobe InDesign or one of its sisters is the right tool for you. But if you don't want to have its drawbacks like bad stability, compitability, and especially lots of mouse clicks, …, WYSIWIM and Markup Creator could be the right tool. For instance, in science and engineering (this is the target audience of Markup Creator, too) LaTeX is very popular because of the powerful math support and render quality. MS Word slowly became competitor in this aspect, but if you want to have equations, you have to do an enormous amount of mouse klicks – typing TeX-math is way faster! There are even people out there doing things like http://mirrors.ctan.org/macros/latex/contrib/chemfig/chemfig_doc_en.pdf or http://mirror.ctan.org/macros/generic/diagrams/circ/circ.pdf. LaTeX documents in general have a very professional look partly for equations, because of the thoroughly developed typography over the years. That's the reason why many books on science and engineering are written with LaTeX.

There are LaTeX editors, but many of them (e.g. TeXnicCenter) are not cross platform and miss features which I want to implement in Markup Creator. LaTeX itself has huge drawbacks: XML and CSS is a much better concept if you want to design your document layout by yourself – to use XML and CSS for paged documents is possible since Prince XML is there, which is great! But there is no nice and simple XML and CSS editor: auto completion, outline, syntax styling (not just highlighting) and some other features.

Editing markup by hand is a terrible […]

If the editor you use does not support you, absolutely yes! That's my motivation on Markup Creator. At the end you should be at least as fast as with WYSIWIG-tools – if not this project has fialed. I'm also planning to make a formatting tool bar like this (this is from Apple Pages):
formatToolBar.png

...and a shortcut-insertion feature: e.g. with the shortcut CTRL+I [an insert bar appears with "buttons" to click or faster type further a key] h [for headings, next buttons] 1 [for h1] [<h1></h1> inserted]. For more complex things like a bibliography item, I imagine a question-answere-conversion similar to emacs command line.

The markup language can support you, too: HAML and maybe pwilang http://code.google.com/p/pwilang/ will be fully supported.

but once you've made a design decision about storage, it's mad to force the user to pretend to be a computer

I don't do any design decision on storage. You could say LaTeX does or the LaTeX document class does it respectively. But in XML (or HAML) the user does and this could change with every new document if you want to – that's the cool thing on XML. I don't think it is a drawback: If you want to change e.g. all <h1> to <heading1>, you could do a search&replace.

The goal is to write and edit documents with WYSIWYM as easy and as fast as possible.
  1. formatToolBar.png12.5 kB
    Format tool bar
Nicholas
If speed is the idea, then think again if you think keystrokes based on manually entering serialised markup is the best way. I don't care about the mouse; I didn't have one for years, and that's not what I'm talking about. There is a glut of text editors using unstructured input of one sort or another to edit the unstructured serialisation of structured data; before just assuming that markup editing is sensible, I was just suggesting you have a rethink of your UI assumptions.

In science and engineering, we hate LaTeX because it is fundamentally the wrong way to go about working with your data, but for legacy reasons and decades of fine typographical tuning we have to put up with it. If we had to start from scratch, there is no way we would make anything with a UI even near to the old paradigm.

I suggest again that you look at XXE, rather than thinking that moving away from directly editing your file format forces you into an interface like Word, OO, InDesign which you mention but I was not talking about. I don't want to hold one piece of software up as a model, because there are some big things I think it could improve on, but it is good to challenge our UI assumptions.

Auto completion and syntax styling are tiny drops in the bucket compared to the efficiency and UI fluency gains you could be getting by trying something other than markup editing. You're kind of in the awkward gap between realising that markup is not helpful to the user and then going on to really doing something about it. Think about it again: suppose you want to enter a heading. The heading is stored on disk as the characters "<h1>", but the internals of the file format are not of concern to the user; they should only need to know the semantic model of the document. If you are starting from scratch, it's clear that there is no reason on the face of it that the user's interface to his editor must necessarily involve him typing the "<h" himself, and if you can find something which saves two keystrokes (like CTL-H then 1) it would be pretty stupid to display the "<h1>" to the user given that he didn't even type it. In fact, you're approaching it from the other end, saying that displaying an <h1> tag is not a good enough user interface; why not apply some styling to the text to colour it and make it bigger. You're right: that's a much more natural way of representing a heading, and if that's the case, then again, why clutter your UI with some legacy junk by showing the user the "<h1>" strings? Sure, that's how it will be saved, but we have to stop thinking of the file format as an interface for the user. Think bigger, and don't just use syntax styling, but go one further and don't bother displaying the tags at all; the software can handle the XML syntax and closing the tags better than the user, who shouldn't have to worry about it. Do you get what I am suggesting? Thinking about UI carefully ought to convince you that markup editing is not the right UI for editing HTML documents. If you like the idea of WYSIWYM, then stop and think why it is that the file format internals should be what you see. We don't need to show those, and we can do something better instead, and when we stop tying down the keyboard input to the markup, you can make that faster too.

On a side-note, if you want cross-platform, it makes most sense to do that in the browser. A browser-based editor with a tiny PHP shim would work just fine to open and edit easily all the documents on your machine, and would make your software relevant to the main application of HTML editors, for editing content to go up on websites, with PDF as the option for printing.
StoneCypher
Won't resolve. :(

John Haugeland is http://fullof.bs/