Bug Report: Oxygen thinks DocBook <reference> is DITA <reference>
Hello to Oxygen Support, with thanks for such an excellent product. There is a small bug in a reproducible corner case, where Oxygen gets confused by a DocBook 4 <reference> element used at the top level of a file, and tries to validate the file in the DITA context. I’m attaching a zip file containing example files. This was tested with Oxygen 14, but I’ve seen this behavior in previous releases. Steps to reproduce: 1. Create a new DocBook 4 file from the article template in Oxygen. 2. In the new file, replace “DOCTYPE article” with “DOCTYPE reference”, and change both instances of 4.5 to 4.4 (to match the standard we still use). 3. In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html. 4. Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string. 5. A file as described above is in the attached zip file as def-guide-reference-example-1.xml. 6. This file validates as a DocBook file in Oxygen, as expected, as well as in XXE (XMLMind XML Editor) and with xmllint. But wait, there’s more: 7. Now create two more new DocBook 4.4 files as described above. Change to “DOCTYPE refentry” and replace the <article> contents with the two <refentry> elements from the first example. 8. Two files as described here are in the zip file as awk.xml and gnu-as.xml. 9. Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml. 10. This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files. 11. As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that’s not the problem. Notice the validation failure messages, which begin as follows: [cid:image001.png@01CD6E3C.66BF2B30] 12. Even if you edit this second example to add Oxygen’s xinclude.mod support, the file is still considered to be in the DITA context. 13. Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above. 14. Comment out the XIncludes, and Oxygen validation returns to the DocBook context. Conclusion: there is something about the combination of a <reference> top-level element and an <xi:include> element that makes Oxygen ignore the DOCTYPE declaration and instead treat the file as a DITA file. Note again that XXE and xmllint* have no problem with this combination. We grudgingly accept the fact that Oxygen does not support XIncludes in DocBook 4 files unless the xinclude.mod kludge is used. But Oxygen should never consider an otherwise valid DocBook-declared file as a DITA file, despite the use of the <reference> element in both standards. *Our xmllint validation command is: xmllint --xinclude --catalogs --timing --noout --noent –postvalid filename.xml You can ignore a message that SGML_CATALOG_FILES is not set, or just set that to =/etc/xml/catalog. (Tested on Cygwin 1.7 and Linux Mint 11.)
Attached zip file actually attached this time. From: oxygen-user-bounces@oxygenxml.com [mailto:oxygen-user-bounces@oxygenxml.com] On Behalf Of Barton Wright Sent: Monday, July 30, 2012 10:17 AM To: oxygen-user@oxygenxml.com Subject: [oXygen-user] Bug Report: Oxygen thinks DocBook <reference> is DITA <reference> Hello to Oxygen Support, with thanks for such an excellent product. There is a small bug in a reproducible corner case, where Oxygen gets confused by a DocBook 4 <reference> element used at the top level of a file, and tries to validate the file in the DITA context. I’m attaching a zip file containing example files. This was tested with Oxygen 14, but I’ve seen this behavior in previous releases. Steps to reproduce: 1. Create a new DocBook 4 file from the article template in Oxygen. 2. In the new file, replace “DOCTYPE article” with “DOCTYPE reference”, and change both instances of 4.5 to 4.4 (to match the standard we still use). 3. In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html. 4. Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string. 5. A file as described above is in the attached zip file as def-guide-reference-example-1.xml. 6. This file validates as a DocBook file in Oxygen, as expected, as well as in XXE (XMLMind XML Editor) and with xmllint. But wait, there’s more: 7. Now create two more new DocBook 4.4 files as described above. Change to “DOCTYPE refentry” and replace the <article> contents with the two <refentry> elements from the first example. 8. Two files as described here are in the zip file as awk.xml and gnu-as.xml. 9. Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml. 10. This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files. 11. As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that’s not the problem. Notice the validation failure messages, which begin as follows: [cid:image001.png@01CD6E3D.4CD8CEF0] 12. Even if you edit this second example to add Oxygen’s xinclude.mod support, the file is still considered to be in the DITA context. 13. Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above. 14. Comment out the XIncludes, and Oxygen validation returns to the DocBook context. Conclusion: there is something about the combination of a <reference> top-level element and an <xi:include> element that makes Oxygen ignore the DOCTYPE declaration and instead treat the file as a DITA file. Note again that XXE and xmllint* have no problem with this combination. We grudgingly accept the fact that Oxygen does not support XIncludes in DocBook 4 files unless the xinclude.mod kludge is used. But Oxygen should never consider an otherwise valid DocBook-declared file as a DITA file, despite the use of the <reference> element in both standards. *Our xmllint validation command is: xmllint --xinclude --catalogs --timing --noout --noent –postvalid filename.xml You can ignore a message that SGML_CATALOG_FILES is not set, or just set that to =/etc/xml/catalog. (Tested on Cygwin 1.7 and Linux Mint 11.)
Dear Barton, Thanks for the details and the samples. In the Oxygen "Preferences" dialog there is a "Document Type Association" page. When an XML document is opened, Oxygen needs to determine what type of document it is. This is useful for many things like providing a default CSS to edit the document in the Author page or providing extension actions in the Author page. So it does this by asking each of the document types in the "Document Type Association" list if it recognizes that document. If you edit the "DITA" document type association you will notice that it has a list of "Association rules". One of those associations is made by checking that the root element name is "reference". So I think this is the main problem, because of this specific rule Oxygen considers that your Docbook 4 <reference>'s are matched by the DITA document type. But if the XML documents have DOCTYPE's (which they do) Oxygen will use those DOCTYPE's when validating so it will not validate them against the DITA DTDs. You should try to remove that specific rule from the DITA document type, there are other, smarter rules in the same list which will match the DITA references. We will also make this correction on our side. If you do not need "DITA" you can disable the document type completely (along with DITA Map) in the list of document types. To see that you have succeeded, in the "Window" menu there is a "Show view" submenu which can be used to show a view called "Properties". The properties view shows the document type which is associated with the selected XML document. Please see more comments to your observations below:
1. Create a new DocBook 4 file from the article template in Oxygen. 2. In the new file, replace “DOCTYPE article” with “DOCTYPE reference”, and change both instances of 4.5 to 4.4 (to match the standard we still use). 3. In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html. 4. Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string. ........................
One productivity remark: You can add additional new file templates to the ones which come with Oxygen in order to faster create new types of XML documents (like Docbook <reference>):
http://www.oxygenxml.com/doc/ug-oxygen/index.html?q=/doc/ug-oxygen/topics/pr...
Basically all Docbook 4 file templates reside here:
OXYGEN_INSTALL_DIR\frameworks\docbook\templates\Docbook 4
so you can either add new files to that folder or follow the explanations in the link above to add a new templates directory to Oxygen.
7. Now create two more new DocBook 4.4 files as described above. Change to “DOCTYPE refentry” and replace the <article> contents with the two <refentry> elements from the first example. 8. Two files as described here are in the zip file as awk.xml and gnu-as.xml. 9. Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml. 10. This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files. 11. As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that’s not the problem. Notice the validation failure messages, which begin as 12. Even if you edit this second example to add Oxygen’s xinclude.mod support, the file is still considered to be in the DITA context. 13. Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above. 14. Comment out the XIncludes, and Oxygen validation returns to the DocBook context.
By default Oxygen uses the Xerces parser for validation and Xerces validates XML files which have DTDs associated before the xi:include is expanded. LibXML validates the file after the xi:included sections have been expanded. You can also use LibXML for validation in Oxygen. In order to validate Docbook 4 <reference> files which contain <xi:include> tags using the default Xerces parser the DOCTYPE should include the xi:include module like:
<!DOCTYPE reference PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" "http://www.docbook.org/xml/4.4/docbookx.dtd"[ <!ENTITY % xinclude SYSTEM "http://www.docbook.org/xml/4.4/xinclude.mod" > %xinclude; <!ENTITY % local.common.attrib "xml:base CDATA #IMPLIED" > ]>
In addition to this you should edit the file:
OXYGEN_INSTALL_DIR/frameworks/docbook/4.4/dtd/xinclude.mod
and add an additional entry to it:
<!ENTITY % local.refentry.class "| xi:include" >
Hope this helps. Regards, Radu Radu Coravu <oXygen/> XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com On 7/30/2012 5:16 PM, Barton Wright wrote:
Hello to Oxygen Support, with thanks for such an excellent product.
There is a small bug in a reproducible corner case, where Oxygen gets confused by a DocBook 4 <reference> element used at the top level of a file, and tries to validate the file in the DITA context. I’m attaching a zip file containing example files. This was tested with Oxygen 14, but I’ve seen this behavior in previous releases.
Steps to reproduce:
1.Create a new DocBook 4 file from the article template in Oxygen.
2.In the new file, replace “DOCTYPE article” with “DOCTYPE reference”, and change both instances of 4.5 to 4.4 (to match the standard we still use).
3.In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html.
4.Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string.
5.A file as described above is in the attached zip file as def-guide-reference-example-1.xml.
6.This file validates as a DocBook file in Oxygen, as expected, as well as in XXE (XMLMind XML Editor) and with xmllint.
But wait, there’s more:
7.Now create two more new DocBook 4.4 files as described above. Change to “DOCTYPE refentry” and replace the <article> contents with the two <refentry> elements from the first example.
8.Two files as described here are in the zip file as awk.xml and gnu-as.xml.
9.Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml.
10.This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files.
11.As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that’s not the problem. Notice the validation failure messages, which begin as follows:
12.Even if you edit this second example to add Oxygen’s xinclude.mod support, the file is still considered to be in the DITA context.
13.Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above.
14.Comment out the XIncludes, and Oxygen validation returns to the DocBook context.
*Conclusion*: there is something about the combination of a <reference> top-level element and an <xi:include> element that makes Oxygen ignore the DOCTYPE declaration and instead treat the file as a DITA file.
Note again that XXE and xmllint* have no problem with this combination.
We grudgingly accept the fact that Oxygen does not support XIncludes in DocBook 4 files unless the xinclude.mod kludge is used. But Oxygen should never consider an otherwise valid DocBook-declared file as a DITA file, despite the use of the <reference> element in both standards.
*Our xmllint validation command is:
xmllint --xinclude --catalogs --timing --noout --noent –postvalid /filename./xml
You can ignore a message that SGML_CATALOG_FILES is not set, or just set that to =/etc/xml/catalog. (Tested on Cygwin 1.7 and Linux Mint 11.)
_______________________________________________ oXygen-user mailing list oXygen-user@oxygenxml.com http://www.oxygenxml.com/mailman/listinfo/oxygen-user
But if the XML documents have DOCTYPEs (which they do) Oxygen will use those DOCTYPEs when validating so it will not validate them against the DITA DTDs.
Hmm. So by default in this case, Oxygen validates against the DocBook 4 DTD, but otherwise still treats the file as if it were a DITA <reference> file, such as when switching to Author mode. That's a bit of a Frankenstein monster, isn't it? I have implemented your Doc Type Association workaround, and it works well, but it strikes me as only a workaround. I don't want to break DITA support for us or for other users. A genuine solution would be for Oxygen to honor the DOCTYPE over the top-level element for file type identification as well as for validation, wouldn't it?
You can add additional new file templates ...
Yes, thanks for the reminder. If Oxygen was our primary XML editor, we would take advantage of all the productivity shortcuts available.
You can also use LibXML for validation in Oxygen.
This was great information. We had not explored the custom validation feature enough, and we're delighted that you pointed it out.
In order to validate Docbook 4 <reference> files which contain <xi:include> tags using the default Xerces parser the DOCTYPE should include the xi:include module ...
Yes, that's what my note referred to as "the xinclude.mod kludge". For historical and technical reasons, our doc set does not use any entities anywhere in our DocBook source files. (We use Unicode fonts and UTF-8 file encoding instead of character entities, and use XInclude instead of file or fragment entities.) Thus, importing xinclude.mod as a file entity is not available to us. Thanks for your detailed response. -----Original Message----- From: oxygen-user-bounces@oxygenxml.com [mailto:oxygen-user-bounces@oxygenxml.com] On Behalf Of Radu Coravu Sent: Tuesday, July 31, 2012 2:21 AM To: oxygen-user@oxygenxml.com Subject: Re: [oXygen-user] Bug Report: Oxygen thinks DocBook <reference> is DITA <reference> Dear Barton, Thanks for the details and the samples. In the Oxygen "Preferences" dialog there is a "Document Type Association" page. When an XML document is opened, Oxygen needs to determine what type of document it is. This is useful for many things like providing a default CSS to edit the document in the Author page or providing extension actions in the Author page. So it does this by asking each of the document types in the "Document Type Association" list if it recognizes that document. If you edit the "DITA" document type association you will notice that it has a list of "Association rules". One of those associations is made by checking that the root element name is "reference". So I think this is the main problem, because of this specific rule Oxygen considers that your Docbook 4 <reference>'s are matched by the DITA document type. But if the XML documents have DOCTYPE's (which they do) Oxygen will use those DOCTYPE's when validating so it will not validate them against the DITA DTDs. You should try to remove that specific rule from the DITA document type, there are other, smarter rules in the same list which will match the DITA references. We will also make this correction on our side. If you do not need "DITA" you can disable the document type completely (along with DITA Map) in the list of document types. To see that you have succeeded, in the "Window" menu there is a "Show view" submenu which can be used to show a view called "Properties". The properties view shows the document type which is associated with the selected XML document. Please see more comments to your observations below:
1. Create a new DocBook 4 file from the article template in Oxygen. 2. In the new file, replace "DOCTYPE article" with "DOCTYPE reference", and change both instances of 4.5 to 4.4 (to match the standard we still use). 3. In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html. 4. Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string. ........................
One productivity remark: You can add additional new file templates to the ones which come with Oxygen in order to faster create new types of XML documents (like Docbook <reference>):
http://www.oxygenxml.com/doc/ug-oxygen/index.html?q=/doc/ug-oxygen/top ics/preferences-editor-document-templates.html
Basically all Docbook 4 file templates reside here:
OXYGEN_INSTALL_DIR\frameworks\docbook\templates\Docbook 4
so you can either add new files to that folder or follow the explanations in the link above to add a new templates directory to Oxygen.
7. Now create two more new DocBook 4.4 files as described above. Change to "DOCTYPE refentry" and replace the <article> contents with the two <refentry> elements from the first example. 8. Two files as described here are in the zip file as awk.xml and gnu-as.xml. 9. Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml. 10. This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files. 11. As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that's not the problem. Notice the validation failure messages, which begin as 12. Even if you edit this second example to add Oxygen's xinclude.mod support, the file is still considered to be in the DITA context. 13. Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above. 14. Comment out the XIncludes, and Oxygen validation returns to the DocBook context.
By default Oxygen uses the Xerces parser for validation and Xerces validates XML files which have DTDs associated before the xi:include is expanded. LibXML validates the file after the xi:included sections have been expanded. You can also use LibXML for validation in Oxygen. In order to validate Docbook 4 <reference> files which contain <xi:include> tags using the default Xerces parser the DOCTYPE should include the xi:include module like:
<!DOCTYPE reference PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN" "http://www.docbook.org/xml/4.4/docbookx.dtd"[ <!ENTITY % xinclude SYSTEM "http://www.docbook.org/xml/4.4/xinclude.mod" > %xinclude; <!ENTITY % local.common.attrib "xml:base CDATA #IMPLIED" > ]>
In addition to this you should edit the file:
OXYGEN_INSTALL_DIR/frameworks/docbook/4.4/dtd/xinclude.mod
and add an additional entry to it:
<!ENTITY % local.refentry.class "| xi:include" >
Hope this helps. Regards, Radu Radu Coravu <oXygen/> XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com On 7/30/2012 5:16 PM, Barton Wright wrote:
Hello to Oxygen Support, with thanks for such an excellent product.
There is a small bug in a reproducible corner case, where Oxygen gets confused by a DocBook 4 <reference> element used at the top level of a file, and tries to validate the file in the DITA context. I'm attaching a zip file containing example files. This was tested with Oxygen 14, but I've seen this behavior in previous releases.
Steps to reproduce:
1.Create a new DocBook 4 file from the article template in Oxygen.
2.In the new file, replace "DOCTYPE article" with "DOCTYPE reference", and change both instances of 4.5 to 4.4 (to match the standard we still use).
3.In the new file, replace the entire <article> contents with the contents of the <reference> example from the DocBook Definitive Guide at http://docbook.org/tdg/en/html/reference.html.
4.Add a <referenceinfo> element above the <title> element, containing one <corpauthor> element that contains any string.
5.A file as described above is in the attached zip file as def-guide-reference-example-1.xml.
6.This file validates as a DocBook file in Oxygen, as expected, as well as in XXE (XMLMind XML Editor) and with xmllint.
But wait, there's more:
7.Now create two more new DocBook 4.4 files as described above. Change to "DOCTYPE refentry" and replace the <article> contents with the two <refentry> elements from the first example.
8.Two files as described here are in the zip file as awk.xml and gnu-as.xml.
9.Now replace the <refentry> elements in the main file with XInclude references to the two <refentry> files. A file as described here is in the zip file as def-guide-reference-example-2.xml.
10.This second example validates fine in XXE and with xmllint, both of which can resolve XIncludes in DocBook 4 files.
11.As expected, this file fails to validate in Oxygen because Oxygen cannot natively resolve XIncludes in DocBook 4 files. (We are used to this fact of life, and wish it were otherwise.) But that's not the problem. Notice the validation failure messages, which begin as follows:
12.Even if you edit this second example to add Oxygen's xinclude.mod support, the file is still considered to be in the DITA context.
13.Further, copy a valid <refentry> element back into the file so that it has one <refentry> local to the file and two XIncluded <refentry> pages, and validation still fails with the message shown above.
14.Comment out the XIncludes, and Oxygen validation returns to the DocBook context.
*Conclusion*: there is something about the combination of a <reference> top-level element and an <xi:include> element that makes Oxygen ignore the DOCTYPE declaration and instead treat the file as a DITA file.
Note again that XXE and xmllint* have no problem with this combination.
We grudgingly accept the fact that Oxygen does not support XIncludes in DocBook 4 files unless the xinclude.mod kludge is used. But Oxygen should never consider an otherwise valid DocBook-declared file as a DITA file, despite the use of the <reference> element in both standards.
*Our xmllint validation command is:
xmllint --xinclude --catalogs --timing --noout --noent -postvalid /filename./xml
You can ignore a message that SGML_CATALOG_FILES is not set, or just set that to =/etc/xml/catalog. (Tested on Cygwin 1.7 and Linux Mint 11.)
_______________________________________________ oXygen-user mailing list oXygen-user@oxygenxml.com http://www.oxygenxml.com/mailman/listinfo/oxygen-user
_______________________________________________ oXygen-user mailing list oXygen-user@oxygenxml.com http://www.oxygenxml.com/mailman/listinfo/oxygen-user
Hello, Barton Wright wrote:
But if the XML documents have DOCTYPEs (which they do) Oxygen will use those DOCTYPEs when validating so it will not validate them against the DITA DTDs.
Hmm. So by default in this case, Oxygen validates against the DocBook 4 DTD, but otherwise still treats the file as if it were a DITA <reference> file, such as when switching to Author mode. That's a bit of a Frankenstein monster, isn't it?
I have implemented your Doc Type Association workaround, and it works well, but it strikes me as only a workaround. I don't want to break DITA support for us or for other users. A genuine solution would be for Oxygen to honor the DOCTYPE over the top-level element for file type identification as well as for validation, wouldn't it?
It is not a workaround. It is a correction of the reported DITA/DocBook conflict based on a simplification of the definition of Oxygen's DITA framework. All the DITA files will continue to be recognized and validated properly after this fix which we plan to implement in Oxygen.
In order to validate Docbook 4 <reference> files which contain <xi:include> tags using the default Xerces parser the DOCTYPE should include the xi:include module ...
Yes, that's what my note referred to as "the xinclude.mod kludge". For historical and technical reasons, our doc set does not use any entities anywhere in our DocBook source files. (We use Unicode fonts and UTF-8 file encoding instead of character entities, and use XInclude instead of file or fragment entities.) Thus, importing xinclude.mod as a file entity is not available to us.
Unfortunately for the default parser which is Xerces there is no other way of declaring the XInclude elements when a DTD is used for validation and the DTD does not include the declaration of the XInclude elements and attributes. This is why the LibXML parser will be better for you as Radu suggested. Best regards, Sorin http://www.oxygenxml.com
participants (3)
-
Barton Wright -
Radu Coravu -
Sorin Ristache