Hello,

In my case only test submodule works others are shown as empty. Web Author remembers the list of sub-modules for 8 hours, it doesn't check for new sub-modules on each request. Did you push these sub-modules recently? Does the issue persist after you restart Web Author?

We'll decrease the amount of time Web Author remembers sub-modules in the next version of Web Author. Best, Gabriel Gabriel Titerlea https://www.oxygenxml.com On 28-Nov-19 12:28, Jirka Kosek wrote:

On 28.11.2019 10:52, oXygen XML Editor Support (Gabriel Titerlea) wrote:

...

...

if there are more submodules added into one repo then only first submodule works correctly. When you say more submodules do you mean nested submodules, where a submodule includes another submodule of its own? Web Author cannot open nested submodules at the moment. I'm not using nested submodules. Are submodules are only added at the top level of repo.

...

Otherwise, Web Author should be able to read/write to GitLab submodules. See this repository [1] opened in Web Author [2]. If this is your case can you share an obfuscated version of your .gitmodules file so we can debug this further? [submodule "test"] path = test url = https://example.com/avitextbook/test.git [submodule "010"] path = 010 url = https://example.com/avitextbook/010.git [submodule "test2"] path = test2 url = https://example.com/avitextbook/test2.git

In my case only test submodule works others are shown as empty.

We are using GitLab on-premise as a git repo if it matters.

Many thanks in advance,

Jirka

Hello,

Would it be possible to stay in the main repository even when working with submodules? In the use-cases we encountered the parent repository would have references to its sub-modules, but sub-modules would not have any reference to the parent or other sub-modules. And given that Git repositories can live as independent entities, it would not seem very intuitive to create references to an other Git repository that may or may not be available, depending on whether the repository was referenced as a sub-module.

We're trying to justify a change to the current behavior regarding Git sub-modules in Web Author. Please describe your use-case in more detail. Best, Gabriel Gabriel Titerlea https://www.oxygenxml.com On 02-Dec-19 16:47, Jirka Kosek wrote:

On 28.11.2019 14:49, oXygen XML Editor Support (Gabriel Titerlea) wrote:

...

...

In my case only test submodule works others are shown as empty. Web Author remembers the list of sub-modules for 8 hours, it doesn't check for new sub-modules on each request. Did you push these sub-modules recently? Does the issue persist after you restart Web Author? Hi,

yep that did the trick -- another day there was no problem accessing content of submodules. Thank you very much.

I have encountered another issue with submodules. Let's consider the following layout of repositories.

project/test1 - git repository with some assets and files project/test2 - another git repository with some assets and files

project/main - git repository that adds test1 and test2 as a top level submodules in folders test1 and test2

Now I create some XML file inside project/main repository -- either at the top level or inside some folder (that is real folder inside project/main not folder attached to submodule). Now I'm using actions from toolbar for inserting images (for example for DocBook) -- file picker dialog allows me to choose any files including files in submodules. When I choose image from submodule it's correctly referenced, e.g.:

<imagedata fileref="test1/images/logo.png"/>

However if I create XML file in folder attached to submodule Web Author automatically switches to submodule repository and it's no longer possible to reference images from the main repository or from other submodules. Respectively it's possible if I change repository manually but URL of referenced image is unusable as it is no longer relative reference, e.g.:

<imagedata fileref="gitgle://https%3A%2F%2Fexample.com%2Fproject%2Fmain/master/test1/images/logo.png"/>

This is probably caused by the fact that when opening/creating file in folder attached to submodule Web Author switches to submodule repository automatically. So in my example setup when I'm trying to work with file

test1/example.xml in project/main repository

Web Author automatically uses

example.xml in project/test1 repository

Would it be possible to stay in the main repository even when working with submodules?

Many thanks in advance,

Jirka

Hello, Regarding Web Author, we'll retain the current behavior regarding git sub-modules for now. But for your use-case we think allowing users to switch the file-browser to the parent repository and choosing a file from there is a good compromise. To make this work you will need to add custom code that transforms absolute URLs into relative URLs and vice versa while taking into account sub-modules. To do that there are 3 hooks where you should intervene using a custom Web Author plugin [1][2]. 1. Add a URI resolver [3] which will be used to resolve relative paths to absolute paths. 2. Add a Relative Reference Resolver which will be used to resolve absolute paths to relative paths on the server side. 3. Add a Relative Reference Resolver which will be used to resolve absolute paths to relative paths on the client side. I'm attaching an example for points 1 & 2: package example.plugin import java.net.MalformedURLException; import java.net.URI; import java.net.URISyntaxException; import java.net.URL; import java.util.Optional; import java.util.function.Predicate; import java.util.regex.Matcher; import java.util.regex.Pattern; import javax.xml.transform.Source; import javax.xml.transform.TransformerException; import javax.xml.transform.URIResolver; import ro.sync.basic.util.URLUtil; import ro.sync.exml.plugin.workspace.WorkspaceAccessPluginExtension; import ro.sync.exml.workspace.api.standalone.StandalonePluginWorkspace; public class MyWorkspaceAccessPluginExtension implements WorkspaceAccessPluginExtension { /** * Predicate to check if a URL is inside a configured parent repository. * You can hard-code this, save it in an option or determine it dynamically depending on your use-case. * */ private Predicate<String> urlIsInsideParentRepository = Pattern.compile("^gitgle://([^@]+@)?https%3A%2F%2Fgitlab.example.com%2Fowner%2Fparent-project/master/").asPredicate(); /** * This regExp extract first part of a Git URL, without the path part. * More details about Git URLs: https://www.oxygenxml.com/doc/versions/21.1.1.0/ug-waCustom/topics/oxy-url.h... * * In this example I'm looking at `gitgle` URLs which are URLs for GitLab on-premise. */ private Pattern gitUrlWithoutPathPattern = Pattern.compile("^gitgle://([^@]+@)?([^:/]+)(:\\d+)?/[^/]+/"); /** * The URL of the submodule inside a parent repository. */ private String submoduleUrlInsideParent = "gitgle://https%3A%2F%2Fgitlab.example.com%2Fowner%2Fparent-project/master/submodule/"; URIResolver uriResolver = new URIResolver() { @Override public Source resolve(String href, String base) throws TransformerException { try { Optional<String> pathRelativeToParentRepo = getPathRelativeToSubmoduleInParentRepo(href, base); if (pathRelativeToParentRepo.isPresent()) { URI uriInParentRepo = new URI(submoduleUrlInsideParent + pathRelativeToParentRepo.get()); URL normalizedUrlInParentRepo = uriInParentRepo.normalize().toURL(); /* * Attach session information to the URL. * This is needed to authenticate the requests that will retrieve the content of this URL. * */ URL hrefWithUserInfo = URLUtil.attachUserInfo(normalizedUrlInParentRepo, new URL(base).getUserInfo(), null, false); return new Source() { @Override public void setSystemId(String systemId) {/**/} @Override public String getSystemId() { return hrefWithUserInfo.toExternalForm(); } }; } } catch (URISyntaxException | MalformedURLException e) {/**/} return null; } /** * Returns a href that is relative to the submodule from the parent URL if the given href represents a navigation outside of the submodule. * @param href An href attribute, which may be relative or absolute. * @param base The base URI against which the first argument will be made absolute if the absolute URI is required. * @return A href that is relative to the parent URL (The repository that contains submodules). * @throws URISyntaxException */ private Optional<String> getPathRelativeToSubmoduleInParentRepo(String href, String base) throws URISyntaxException { Optional<String> relativePathToParentRepo = Optional.empty(); // If the URL is not inside a submodule there's no need to make any change to it. if (base != null && href != null /*&& urlIsInsideSubmodule.test(base)*/) { // If the href is absolute we leave it as is. if (!new URI(href).isAbsolute()) { String baseUrlWithoutFileName = base.substring(0, base.lastIndexOf('/') + 1); /* * ______________ ______ ____________ * A git URL looks like this: gitgle://REPOSITORY_URI/BRANCH/PATH/TO/FILE * * We want to append our relative href to the base git URL and normalize it. * But we need to add an extra "../" to account for the /BRANCH/ part of the git URL. * */ URI normalized = new URI(baseUrlWithoutFileName + "../" + href).normalize(); String absoluteUrlInsideSubmodule = normalized.toString(); int upOneLevelIndex = absoluteUrlInsideSubmodule.indexOf(".."); // After normalization any "../" left represents a navigation outside the current submodule. if (upOneLevelIndex != -1) { String hrefRelativeToParent = absoluteUrlInsideSubmodule.substring(upOneLevelIndex); relativePathToParentRepo = Optional.of(hrefRelativeToParent); } } } return relativePathToParentRepo; } }; @Override public void applicationStarted(StandalonePluginWorkspace pluginWorkspaceAccess) { // Add a URI resolver which will be used to resolve relative paths to absolute paths. pluginWorkspaceAccess.getXMLUtilAccess().addPriorityURIResolver(uriResolver); // Add a Relative Reference Resolver which will be used to resolve absolute paths to relative paths. pluginWorkspaceAccess.addRelativeReferencesResolver("gitgle", (baseUrl, childUrl) -> { try { boolean baseIsInsideAParent = urlIsInsideParentRepository.test(baseUrl.toExternalForm()); boolean childIsInsideAParent = urlIsInsideParentRepository.test(childUrl.toExternalForm()); if (!baseIsInsideAParent && childIsInsideAParent) { // Make the childUrl relative to the baseUrl while taking submodules into account. String baseUrlString = baseUrl.toExternalForm(); Matcher matcher = gitUrlWithoutPathPattern.matcher(baseUrlString); if (matcher.find()) { String submoduleUrl = baseUrlString.substring(matcher.start(), matcher.end()); URL submoduleURL = URLUtil.attachUserInfo(new URL(submoduleUrl), baseUrl.getUserInfo(), null, false); // To make this work for all submodules you could have a map from a submodule URL to its URL inside the parent. URL submoduleInParentURL = URLUtil.attachUserInfo(new URL(submoduleUrlInsideParent), baseUrl.getUserInfo(), null, false); String relativePathToRootOfSubmodule = URLUtil.makeRelative(baseUrl, submoduleURL, true, true); if (relativePathToRootOfSubmodule.equals(".")) { relativePathToRootOfSubmodule = ""; } String relativePathToTargetInsideParent = URLUtil.makeRelative(submoduleInParentURL, childUrl, true, true); // Resolved relative URL return relativePathToRootOfSubmodule + relativePathToTargetInsideParent; } } } catch (MalformedURLException e) { /**/ } return null; }); } @Override public boolean applicationClosing() { return true; } } The example.plugin.MyWorkspaceAccessPluginExtension class need to be referenced in the plugin.xml file of your plugin [4]. For point 2, you need to add a relative references resolver [5] in javascript. This resolver will be used on the client's browser to resolve URLs. (function () { workspace.addRelativeReferencesResolver('gitgle', function (baseUrl, childUrl) { // Re-implement the java code which was used for `pluginWorkspaceAccess.addRelativeReferencesResolver`. return "resolvedUrl"; }); })(); Let us know if you need anything else. Best, Gabriel [1] https://www.oxygenxml.com/doc/versions/21.1.1.0/ug-waCustom/topics/customizi... [2] https://www.oxygenxml.com/doc/versions/21.1.1.0/ug-waCustom/topics/webapp-pl... <https://www.oxygenxml.com/doc/versions/21.1.1.0/ug-waCustom/topics/webapp-plugin-prototyping.html?hl=Go%20to%20the%20following%20repository%20and%20follow%20the%20instructions> [3] https://www.oxygenxml.com/doc/versions/21.1.1.0/ug-waCustom/topics/oxy-url.h... [4] https://www.oxygenxml.com/doc/versions/21.1/ug-editor/topics/workspace-acces... [5] https://www.oxygenxml.com/maven/com/oxygenxml/oxygen-webapp/21.1.1.0/jsdoc/s... Gabriel Titerlea https://www.oxygenxml.com On 03-Dec-19 18:48, Jirka Kosek wrote:

On 3.12.2019 12:40, oXygen XML Editor Support (Gabriel Titerlea) wrote:

...

We're trying to justify a change to the current behavior regarding Git sub-modules in Web Author. Please describe your use-case in more detail. It's project where requirements changed over the time ;-)

Originally separate textbooks have been edited -- each textbook in a separate git repository. Each textbook has been standalone. For each textbook different group of users has access rights.

Now requirements changed and some books need to reuse content (chapters/section, images, ...) from other books. Merging all individual repositories into one large repository (for example using git subtree feature) where there will be separate folder for each book is not viable as this way we would loose ability to give different access rights for different textbooks. Git can set access rights only on whole repository not on individual folders inside it.

So using submodules we can keep textbooks that don't require inclusion of assets from other textbooks as they were in a separate repositories. Then textbooks that need to reuse content can be edited from "grandrepo" that would include all textbooks as submodules.

Well, yes perhaps this is far beyond what git has been designed for. Git was fine for original requirements. Knowing current requirements at the start of project would mean that expensive CCMS should be used instead of git. :-(

Thanks and have a nice day,

Jirka