[Bug report] Create table Hive table failed due to metadata object does not exists #4812
Comments
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics
github-actions bot
pushed a commit
that referenced
this issue
Aug 30, 2024
…es and topics (#4818) ### What changes were proposed in this pull request? If the schema is not created by Gravitino, the Gravitino will lack the metadata in the backend storage. If we create a table in this schema, the storage won't contain the metadata. So it will fail to set owner. Because the storage won't store the table. Because the storage won't contain schema id. This won't bring too much performance cost. Because loadSchema will use read lock after first loading. If we have cache, we could be more quick. ### Why are the changes needed? Fix: #4812 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added a ut.
jerqi
added a commit
to qqqttt123/gravitino
that referenced
this issue
Aug 30, 2024
…g tables and topics (apache#4818) ### What changes were proposed in this pull request? If the schema is not created by Gravitino, the Gravitino will lack the metadata in the backend storage. If we create a table in this schema, the storage won't contain the metadata. So it will fail to set owner. Because the storage won't store the table. Because the storage won't contain schema id. This won't bring too much performance cost. Because loadSchema will use read lock after first loading. If we have cache, we could be more quick. ### Why are the changes needed? Fix: apache#4812 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added a ut.
jerryshao
pushed a commit
that referenced
this issue
Aug 30, 2024
…es and topics (#4826) ### What changes were proposed in this pull request? If the schema is not created by Gravitino, the Gravitino will lack the metadata in the backend storage. If we create a table in this schema, the storage won't contain the metadata. So it will fail to set owner. Because the storage won't store the table. Because the storage won't contain schema id. This won't bring too much performance cost. Because loadSchema will use read lock after first loading. If we have cache, we could be more quick. ### Why are the changes needed? Fix: #4812 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Added a ut.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Version
main branch
Describe what's wrong
Error message and/or stacktrace
Query 20240830_082854_00029_ij9im failed: Failed to operate table(s) [table_01] operation [CREATE] under schema [db1], reason [The metadata object of hive_catalog2.db1.table_01 isn't found]
org.apache.gravitino.exceptions.NoSuchMetadataObjectException: The metadata object of hive_catalog2.db1.table_01 isn't found
at org.apache.gravitino.authorization.OwnerManager.getOwner(OwnerManager.java:178)
at org.apache.gravitino.authorization.OwnerManager.setOwner(OwnerManager.java:71)
at org.apache.gravitino.hook.TableHookDispatcher.createTable(TableHookDispatcher.java:82)
at org.apache.gravitino.catalog.TableNormalizeDispatcher.createTable(TableNormalizeDispatcher.java:78)
at org.apache.gravitino.listener.TableEventDispatcher.createTable(TableEventDispatcher.java:114)
at org.apache.gravitino.server.web.rest.TableOperations.lambda$createTable$2(TableOperations.java:128)
at org.apache.gravitino.lock.TreeLockUtils.doWithTreeLock(TreeLockUtils.java:49)
at org.apache.gravitino.server.web.rest.TableOperations.lambda$createTable$3(TableOperations.java:124)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:712)
at java.base/javax.security.auth.Subject.doAs(Subject.java:439)
at org.apache.gravitino.utils.PrincipalUtils.doAs(PrincipalUtils.java:39)
at org.apache.gravitino.server.web.Utils.doAs(Utils.java:149)
at org.apache.gravitino.server.web.rest.TableOperations.createTable(TableOperations.java:116)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189)
at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:176)
at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400)
at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:256)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:235)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:394)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:358)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:311)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
at org.apache.gravitino.server.authentication.AuthenticationFilter.doFilter(AuthenticationFilter.java:86)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.apache.gravitino.server.web.VersioningFilter.doFilter(VersioningFilter.java:111)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:833)
How to reproduce
db1 is NOT created by Gravitino and created by outer user.
Additional context
No response
The text was updated successfully, but these errors were encountered: