Re: [oXygen-user] Validation of Schema that Imports Multiple XSD Schemas
Hi Adrian, I checked the paths of the files in the project. They were all in the same directory. oXygen_Global_Options.xml contains the system information. I created and validatedRelation_Image_List13Aug12 with XMLSpy. Validation took 1 to 2 seconds. The same file in oXygen already has the description at over 7,500 items and growing! 50% of the 2 CPUs is being used. I obtained the following message from the bottom panel: Description: Loading catalog: file:/C:/Program%20Files/Oxygen%20XML%20Editor%2014/frameworks/catalog.xml I attempted to validate after about an hour< I received an error message that the system had run out of heap space. System ID: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml Main validation file: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml Schema: D:\CytometryML\ACS\xsd\relations_image.xsd Engine name: Saxon-EE 9.4.0.3 Severity: error Description: Java heap space If the problem is the number of schemas imported, the parser can look in each schema and determine if the prefix is used, if not a comment could be placed next to each import element. The user could then erase it or comment it out. This is equivalent to a compiler parser looking for unused code. Thank you. Yours, Bob Leif -----Original Message----- From: Oxygen XML Editor Support [mailto:support@oxygenxml.com] Sent: Monday, August 13, 2012 6:21 AM To: rleif@rleif.com Cc: oXygen-user@oxygenxml.com Subject: Re: [oXygen-user] Validation of Schema that Imports Multiple XSD Schemas Hello Robert, When validating XML files, Oxygen provides a caching mechanism for the schema used for validation, so only the first validation will evaluate the entire schema hierarchy. The following XML validations will use the cached schema. When validating XML schema files, the entire schema hierarchy starting from the current schema is validated every time. However, the validation time that you report seems extreme for an XML Schema validation, even if a large schema hierarchy is validated. Do you have a validation scenario with multiple validation units or are you using the default validation as it is offered by Oxygen? Make sure the schemas are all local. In situations where schemas are imported from remote locations, make sure these remote URIs are resolved through an XML catalog to a local copy. My guess is some of the schemas from your schema hierarchy are requested from a remote location and that's what's causing the huge validation time. To determine if some schemas are retrieved from remote locations, you can enable XML Catalog verbosity in Oxygen from Options > Preferences, XML > XML Catalog. Set "Verbosity" to "All messages". Whenever you validate or perform an Oxygen operation that involves the catalog resolver, the attempts to resolve URIs, systemIDs or namespaces (depending on your Oxygen catalog options) will appear in the Catalogs view from the results panel (at the bottom of the editor window). Note that all attempts at resolving something are listed there, so it's possible you'll see a lot of redundant messages, first some that fail and eventually one that succeeds in resolving the URI. We are not aware of any schema caching mechanisms that Saxon offers in this regard, but we have not investigated this possibility. But, my belief is that the problem you have encountered lies elsewhere. There seems to be a bottleneck somewhere that slows things down significantly and my first guess is that some of the schema files are not immediately accessible. By any chance, have you tried to validate the same schema hierarchy with Saxon from the command line? Note that you'll need a separate Saxon deployment since the one bundled with Oxygen is only licensed to be used from within the application GUI. Regards, Adrian Adrian Buza oXygen XML Editor and Author Support Tel: +1-650-352-1250 ext.202 Fax: +40-251-461482 support@oxygenxml.com http://www.oxygenxml.com Robert Leif wrote:
Since my designs follow object-oriented design principles, my XSD
schemas import multiple XSD schemas, which provide and permit reuse of
many of their simpleTypes and complexTypes. The time for a successful
validation of a schema that solely imports 9 other schemas and does
not have any data-type or element definitions was in excess of 10
minutes. The subsequent validation took 3 minutes. 2.66 out of 4
gigabytes of memory and 50% of the CPU were used. I validated with
Saxon-EE 9.4.0.3 for XSD1.1.
I suspect that the problem is that Saxon validated all of the imported
schemas. Since these schemas had NOT been changed after their previous
validation, this was a total waste of time! Does Saxon have a tool to
check the time of the previous validation against the time of the last
edit? If not, it needs one. There is a potential problem when two
schemas import each other. In this case, it probably pays to validate
both. A quick way to ameliorate this problem is to extend the
validation-well-formatted pull-down to include Validate only this
schema. Since a schema tree structure is needed to perform an
intelligent validation, it would be useful to make this tree available
to the user and to note if any of the imported schemas were not being
used.
Has anyone else observed similar long validation times with another
validator?
Thank you.
Yours truly,
Bob Leif
----------------------------------------------------------------------
--
_______________________________________________
oXygen-user mailing list
oXygen-user@oxygenxml.com
Dear Bob, Since version 14 release we were working on improving the XML Schema 1.1 support. That includes also supporting Xerces as an XML Schema 1.1 engine. Validating your Relation_Image_List13Aug12.xml with Xerces finishes in less than one second while using Saxon I got 36 seconds. Thanks for sending the sample files, we will be able to investigate what happens when we use Saxon for validation and determine if the issue is in how we use Saxon or in Saxon itself. Best Regards, George -- George Cristian Bina <oXygen/> XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com On 8/14/12 9:08 PM, Robert Leif wrote:
Hi Adrian,
I checked the paths of the files in the project. They were all in the same directory. oXygen_Global_Options.xml contains the system information. I created and validatedRelation_Image_List13Aug12 with XMLSpy. Validation took 1 to 2 seconds. The same file in oXygen already has the description at over 7,500 items and growing! 50% of the 2 CPUs is being used. I obtained the following message from the bottom panel: Description: Loading catalog: file:/C:/Program%20Files/Oxygen%20XML%20Editor%2014/frameworks/catalog.xml
I attempted to validate after about an hour< I received an error message that the system had run out of heap space.
System ID: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml
Main validation file: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml
Schema: D:\CytometryML\ACS\xsd\relations_image.xsd
Engine name: Saxon-EE 9.4.0.3
Severity: error
Description: Java heap space
If the problem is the number of schemas imported, the parser can look in each schema and determine if the prefix is used, if not a comment could be placed next to each import element. The user could then erase it or comment it out. This is equivalent to a compiler parser looking for unused code.
Thank you.
Yours,
Bob Leif
-----Original Message-----
From: Oxygen XML Editor Support [mailto:support@oxygenxml.com] <mailto:[mailto:support@oxygenxml.com]>
Sent: Monday, August 13, 2012 6:21 AM
To: rleif@rleif.com <mailto:rleif@rleif.com>
Cc: oXygen-user@oxygenxml.com <mailto:oXygen-user@oxygenxml.com>
Subject: Re: [oXygen-user] Validation of Schema that Imports Multiple XSD Schemas
Hello Robert,
When validating XML files, Oxygen provides a caching mechanism for the schema used for validation, so only the first validation will evaluate the entire schema hierarchy. The following XML validations will use the cached schema.
When validating XML schema files, the entire schema hierarchy starting from the current schema is validated every time. However, the validation time that you report seems extreme for an XML Schema validation, even if a large schema hierarchy is validated. Do you have a validation scenario with multiple validation units or are you using the default validation as it is offered by Oxygen?
Make sure the schemas are all local. In situations where schemas are imported from remote locations, make sure these remote URIs are resolved through an XML catalog to a local copy. My guess is some of the schemas from your schema hierarchy are requested from a remote location and that's what's causing the huge validation time.
To determine if some schemas are retrieved from remote locations, you can enable XML Catalog verbosity in Oxygen from Options > Preferences, XML > XML Catalog. Set "Verbosity" to "All messages". Whenever you validate or perform an Oxygen operation that involves the catalog resolver, the attempts to resolve URIs, systemIDs or namespaces (depending on your Oxygen catalog options) will appear in the Catalogs view from the results panel (at the bottom of the editor window).
Note that all attempts at resolving something are listed there, so it's possible you'll see a lot of redundant messages, first some that fail and eventually one that succeeds in resolving the URI.
We are not aware of any schema caching mechanisms that Saxon offers in this regard, but we have not investigated this possibility. But, my belief is that the problem you have encountered lies elsewhere. There seems to be a bottleneck somewhere that slows things down significantly and my first guess is that some of the schema files are not immediately accessible.
By any chance, have you tried to validate the same schema hierarchy with Saxon from the command line? Note that you'll need a separate Saxon deployment since the one bundled with Oxygen is only licensed to be used from within the application GUI.
Regards,
Adrian
Adrian Buza
oXygen XML Editor and Author Support
Tel: +1-650-352-1250 ext.202
Fax: +40-251-461482
support@oxygenxml.com <mailto:support@oxygenxml.com>
Robert Leif wrote:
Since my designs follow object-oriented design principles, my XSD
schemas import multiple XSD schemas, which provide and permit reuse of
many of their simpleTypes and complexTypes. The time for a successful
validation of a schema that solely imports 9 other schemas and does
not have any data-type or element definitions was in excess of 10
minutes. The subsequent validation took 3 minutes. 2.66 out of 4
gigabytes of memory and 50% of the CPU were used. I validated with
Saxon-EE 9.4.0.3 for XSD1.1.
I suspect that the problem is that Saxon validated all of the imported
schemas. Since these schemas had NOT been changed after their previous
validation, this was a total waste of time! Does Saxon have a tool to
check the time of the previous validation against the time of the last
edit? If not, it needs one. There is a potential problem when two
schemas import each other. In this case, it probably pays to validate
both. A quick way to ameliorate this problem is to extend the
validation-well-formatted pull-down to include Validate only this
schema. Since a schema tree structure is needed to perform an
intelligent validation, it would be useful to make this tree available
to the user and to note if any of the imported schemas were not being
used.
Has anyone else observed similar long validation times with another
validator?
Thank you.
Yours truly,
Bob Leif
----------------------------------------------------------------------
--
_______________________________________________
oXygen-user mailing list
oXygen-user@oxygenxml.com <mailto:oXygen-user@oxygenxml.com>
_______________________________________________ oXygen-user mailing list oXygen-user@oxygenxml.com http://www.oxygenxml.com/mailman/listinfo/oxygen-user
Hello Bob, We managed to identify this is a Saxon problem. If I validate "Relation_Image_List13Aug12.xml" with Saxon from command line the validation takes about 60 seconds on my computer. We will report this issue to Saxon. Until it is resolved, we can provide you a beta version of oXygen 14.1 that includes also Xerces as an XML Schema 1.1 engine. Regards, Octavian --------------- Octavian Nadolu <oXygen/> XML Editor http://www.oxygenxml.com On 14.08.2012 21:23, George Cristian Bina wrote:
Dear Bob,
Since version 14 release we were working on improving the XML Schema 1.1 support. That includes also supporting Xerces as an XML Schema 1.1 engine. Validating your Relation_Image_List13Aug12.xml with Xerces finishes in less than one second while using Saxon I got 36 seconds. Thanks for sending the sample files, we will be able to investigate what happens when we use Saxon for validation and determine if the issue is in how we use Saxon or in Saxon itself.
Best Regards, George -- George Cristian Bina <oXygen/> XML Editor, Schema Editor and XSLT Editor/Debugger http://www.oxygenxml.com
On 8/14/12 9:08 PM, Robert Leif wrote:
Hi Adrian,
I checked the paths of the files in the project. They were all in the same directory. oXygen_Global_Options.xml contains the system information. I created and validatedRelation_Image_List13Aug12 with XMLSpy. Validation took 1 to 2 seconds. The same file in oXygen already has the description at over 7,500 items and growing! 50% of the 2 CPUs is being used. I obtained the following message from the bottom panel: Description: Loading catalog: file:/C:/Program%20Files/Oxygen%20XML%20Editor%2014/frameworks/catalog.xml
I attempted to validate after about an hour< I received an error message that the system had run out of heap space.
System ID: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml
Main validation file: D:\CytometryML\ACS\xsd\Relation_Image_List13Aug12 .xml
Schema: D:\CytometryML\ACS\xsd\relations_image.xsd
Engine name: Saxon-EE 9.4.0.3
Severity: error
Description: Java heap space
If the problem is the number of schemas imported, the parser can look in each schema and determine if the prefix is used, if not a comment could be placed next to each import element. The user could then erase it or comment it out. This is equivalent to a compiler parser looking for unused code.
Thank you.
Yours,
Bob Leif
-----Original Message-----
From: Oxygen XML Editor Support [mailto:support@oxygenxml.com] <mailto:[mailto:support@oxygenxml.com]>
Sent: Monday, August 13, 2012 6:21 AM
To: rleif@rleif.com <mailto:rleif@rleif.com>
Cc: oXygen-user@oxygenxml.com <mailto:oXygen-user@oxygenxml.com>
Subject: Re: [oXygen-user] Validation of Schema that Imports Multiple XSD Schemas
Hello Robert,
When validating XML files, Oxygen provides a caching mechanism for the schema used for validation, so only the first validation will evaluate the entire schema hierarchy. The following XML validations will use the cached schema.
When validating XML schema files, the entire schema hierarchy starting from the current schema is validated every time. However, the validation time that you report seems extreme for an XML Schema validation, even if a large schema hierarchy is validated. Do you have a validation scenario with multiple validation units or are you using the default validation as it is offered by Oxygen?
Make sure the schemas are all local. In situations where schemas are imported from remote locations, make sure these remote URIs are resolved through an XML catalog to a local copy. My guess is some of the schemas from your schema hierarchy are requested from a remote location and that's what's causing the huge validation time.
To determine if some schemas are retrieved from remote locations, you can enable XML Catalog verbosity in Oxygen from Options > Preferences, XML > XML Catalog. Set "Verbosity" to "All messages". Whenever you validate or perform an Oxygen operation that involves the catalog resolver, the attempts to resolve URIs, systemIDs or namespaces (depending on your Oxygen catalog options) will appear in the Catalogs view from the results panel (at the bottom of the editor window).
Note that all attempts at resolving something are listed there, so it's possible you'll see a lot of redundant messages, first some that fail and eventually one that succeeds in resolving the URI.
We are not aware of any schema caching mechanisms that Saxon offers in this regard, but we have not investigated this possibility. But, my belief is that the problem you have encountered lies elsewhere. There seems to be a bottleneck somewhere that slows things down significantly and my first guess is that some of the schema files are not immediately accessible.
By any chance, have you tried to validate the same schema hierarchy with Saxon from the command line? Note that you'll need a separate Saxon deployment since the one bundled with Oxygen is only licensed to be used from within the application GUI.
Regards,
Adrian
Adrian Buza
oXygen XML Editor and Author Support
Tel: +1-650-352-1250 ext.202
Fax: +40-251-461482
support@oxygenxml.com <mailto:support@oxygenxml.com>
Robert Leif wrote:
Since my designs follow object-oriented design principles, my XSD
schemas import multiple XSD schemas, which provide and permit reuse of
many of their simpleTypes and complexTypes. The time for a successful
validation of a schema that solely imports 9 other schemas and does
not have any data-type or element definitions was in excess of 10
minutes. The subsequent validation took 3 minutes. 2.66 out of 4
gigabytes of memory and 50% of the CPU were used. I validated with
Saxon-EE 9.4.0.3 for XSD1.1.
I suspect that the problem is that Saxon validated all of the imported
schemas. Since these schemas had NOT been changed after their previous
validation, this was a total waste of time! Does Saxon have a tool to
check the time of the previous validation against the time of the last
edit? If not, it needs one. There is a potential problem when two
schemas import each other. In this case, it probably pays to validate
both. A quick way to ameliorate this problem is to extend the
validation-well-formatted pull-down to include Validate only this
schema. Since a schema tree structure is needed to perform an
intelligent validation, it would be useful to make this tree available
to the user and to note if any of the imported schemas were not being
used.
Has anyone else observed similar long validation times with another
validator?
Thank you.
Yours truly,
Bob Leif
----------------------------------------------------------------------
--
_______________________________________________
oXygen-user mailing list
oXygen-user@oxygenxml.com <mailto:oXygen-user@oxygenxml.com>
_______________________________________________ oXygen-user mailing list oXygen-user@oxygenxml.com http://www.oxygenxml.com/mailman/listinfo/oxygen-user
participants (3)
-
George Cristian Bina -
oXygen XML Editor Support -
Robert Leif