Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7.8 RestHighLevelClient - Composable Index Templates API difficult to use #60750

Open
skombijohn opened this issue Aug 5, 2020 · 15 comments
Open

Comments

@skombijohn
Copy link

@skombijohn skombijohn commented Aug 5, 2020

I currently use the legacy IndexTemplates with the "inheritance" mechanism.
During runtime, my application, will read in plain JSON templates from the filesystem and make PutIndexTemplateRequests out of the content using the standard "source(Map<String, Object>)" API available in all Request subclasses to fill the request with the relevant content.

Now, that those templates are becoming legacy stuff, I started evaluating a migration to composite templates ( I think this is a very good concept!).

Now playing with the JAVA HighLevel API here is not very convenient, as there is no simple way to feed an existing JSON into the request. Basically everything must be provided manually in a constructor of ComposableIndexTemplate:

public ComposableIndexTemplate(List<String> indexPatterns, @Nullable Template template, @Nullable List<String> componentTemplates, @Nullable Long priority, @Nullable Long version, @Nullable Map<String, Object> metadata)

which is kind of difficult to use.

It is not very clear how I create a ComponentTemplate or a IndexTemplate using such a component using that API..
I fully understood the "raw" way with HTTP and JSON, but the API does somehow makes it very difficult.

Maybe the API can be improved to match existing APIs.

Addendum:
Of course using the LowLevelClient works fine.

@elasticmachine
Copy link
Collaborator

@elasticmachine elasticmachine commented Aug 5, 2020

Pinging @elastic/es-core-features (:Core/Features/Java High Level REST Client)

@elasticmachine
Copy link
Collaborator

@elasticmachine elasticmachine commented Aug 5, 2020

Pinging @elastic/es-core-features (:Core/Features/Indices APIs)

@dakrone
Copy link
Member

@dakrone dakrone commented Aug 5, 2020

I totally think we could improve the APIs here, and potentially add a builder perhaps to make it easier to build the templates (instead of having to know everything up front) 👍

@mun-inder-jeet-singh
Copy link

@mun-inder-jeet-singh mun-inder-jeet-singh commented Aug 9, 2020

Hi @dakrone
I wish to take this up.
Kindly confirm if I can start working on this.

@dakrone
Copy link
Member

@dakrone dakrone commented Aug 9, 2020

@mun-inder-jeet-singh go for it! Let me know if I can of any help

@adrian-arapiles
Copy link

@adrian-arapiles adrian-arapiles commented Aug 11, 2020

Hi all,

I have the same issue and I face too that rest high client doesn't have a method for put component template.
Anyways, for put a index template from raw json I use this snippet (in Java and with springboot):

PathMatchingResourcePatternResolver resourceLoader = new PathMatchingResourcePatternResolver();
Resource template = resourceLoader.getResources("/templates/your-template.json");
String name = Objects.requireNonNull(template.getFilename()).split("\\.")[0]; //Name from basename of file, in this case your-template
byte[] stringBytes = Files.readAllBytes(Paths.get(template.getURI()));
PutComponentTemplateRequest request = new PutComponentTemplateRequest().name(name);
ComponentTemplate composableIndexTemplate = new ComponentTemplate(StreamInput.wrap(stringBytes));
request.componentTemplate(composableIndexTemplate);

I hope this can be helpfull.

I currently use the legacy IndexTemplates with the "inheritance" mechanism.
During runtime, my application, will read in plain JSON templates from the filesystem and make PutIndexTemplateRequests out of the content using the standard "source(Map<String, Object>)" API available in all Request subclasses to fill the request with the relevant content.

Now, that those templates are becoming legacy stuff, I started evaluating a migration to composite templates ( I think this is a very good concept!).

Now playing with the JAVA HighLevel API here is not very convenient, as there is no simple way to feed an existing JSON into the request. Basically everything must be provided manually in a constructor of ComposableIndexTemplate:

public ComposableIndexTemplate(List<String> indexPatterns, @Nullable Template template, @Nullable List<String> componentTemplates, @Nullable Long priority, @Nullable Long version, @Nullable Map<String, Object> metadata)

which is kind of difficult to use.

It is not very clear how I create a ComponentTemplate or a IndexTemplate using such a component using that API..
I fully understood the "raw" way with HTTP and JSON, but the API does somehow makes it very difficult.

Maybe the API can be improved to match existing APIs.

Addendum:
Of course using the LowLevelClient works fine.

@mun-inder-jeet-singh
Copy link

@mun-inder-jeet-singh mun-inder-jeet-singh commented Aug 11, 2020

Thanks @dakrone.
I was researching to resolve this and found this code which is present in PutIndexTemplate class.

public PutIndexTemplateRequest settings(Map<String, Object> source) {
try {
XContentBuilder builder = XContentFactory.contentBuilder(XContentType.JSON);
builder.map(source);
settings(Strings.toString(builder), XContentType.JSON);
} catch (IOException e) {
throw new ElasticsearchGenerationException("Failed to generate [" + source + "]", e);
}
return this;`` }`

Do we need to create something like this for ComposableIndexTemplate class ?

@dakrone
Copy link
Member

@dakrone dakrone commented Aug 11, 2020

@mun-inder-jeet-singh I would recommend adding a Builder class inside of the ComposableIndexTemplate class that allows a person to iteratively add the pieces for a template until they call .build() and retrieve the final ComposableIndexTemplate class.

I wouldn't worry too much about adding one that takes a Map<String, Object> for now, as I think that is going to be more error prone.

@antrix190
Copy link

@antrix190 antrix190 commented Aug 13, 2020

Was this already solved? Or can i contribute here? @dakrone @mun-inder-jeet-singh

@dakrone
Copy link
Member

@dakrone dakrone commented Aug 13, 2020

@antrix190 it has not already been solved

@antrix190
Copy link

@antrix190 antrix190 commented Aug 13, 2020

@dakrone I would like to give it a try.

antrix190 added a commit to antrix190/elasticsearch that referenced this issue Aug 13, 2020
@mun-inder-jeet-singh
Copy link

@mun-inder-jeet-singh mun-inder-jeet-singh commented Aug 13, 2020

Hi @antrix190 , I was working on this issue though, this was my first tryout for opensource, hence taking time to complete this issue.

@antrix190
Copy link

@antrix190 antrix190 commented Aug 14, 2020

That's why i asked before picking it up. :)

@antrix190
Copy link

@antrix190 antrix190 commented Aug 16, 2020

Can somebody please review my PR?

antrix190 added a commit to antrix190/elasticsearch that referenced this issue Aug 16, 2020
@mun-inder-jeet-singh
Copy link

@mun-inder-jeet-singh mun-inder-jeet-singh commented Aug 16, 2020

Hi @dakrone,
I have another approach if its feasible.
Can we add lombok dependency here ? Then We just need to add @builder above the constructor of the class.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.