Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

running models and obtaining results #459

Open
rocky2397 opened this issue Oct 7, 2024 · 3 comments
Open

running models and obtaining results #459

rocky2397 opened this issue Oct 7, 2024 · 3 comments

Comments

@rocky2397
Copy link

Hi,
I am currently trying to use GERBIL for the evaluation of some of your models.
(specifically D2KB with mGENRE and our imported model SpEL hosted on our localhost)

We already deactivated the HTTPBasedSameAsRetriever in the gerbil.properties but it still does all the redirects, which makes the evaluation really slow.

Is there a way to fix it or is there maybe a problem with the code, which need to be updated?

Any suggestion would be really helpful at this point.

@MichaelRoeder
Copy link
Member

Hi,
Thank you for using GERBIL.

How do you know that it still looks for the redirects? 🤔

When you run an experiment and you have the impression that GERBIL is too slow, could you please open http://localhost:1234/gerbil/running (please adapt the address as needed), copy the stack trace(s) that you see there and paste them here? That could help a lot to figure out what GERBIL is actually doing 😉

@rocky2397
Copy link
Author

so this is basically what is stated in the running under URL.

eTConfig("SpEL MSNBC (NIF WS)","MSNBC","D2KB","STRONG_ANNOTATION_MATCH")
state=RUNNABLE
progress=null
java.base/sun.nio.ch.Net.poll(Native Method)
java.base/sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:191)
java.base/sun.nio.ch.NioSocketImpl.park(NioSocketImpl.java:201)
java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:309)
java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
java.base/java.net.Socket$SocketInputStream.implRead(Socket.java:1108)
java.base/java.net.Socket$SocketInputStream.read(Socket.java:1095)
java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:489)
java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:483)
java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1462)
java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:1068)
org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
org.aksw.gerbil.dataset.check.impl.HttpBasedEntityChecker.entityExists(HttpBasedEntityChecker.java:64)
org.aksw.gerbil.dataset.check.impl.EntityCheckerManagerImpl.checkUri(EntityCheckerManagerImpl.java:135)
org.aksw.gerbil.dataset.check.impl.FileBasedCachingEntityCheckerManager.checkUri(FileBasedCachingEntityCheckerManager.java:156)
org.aksw.gerbil.dataset.check.impl.EntityCheckerManagerImpl.checkMeaning(EntityCheckerManagerImpl.java:84)
org.aksw.gerbil.dataset.check.impl.EntityCheckerManagerImpl.checkMarkings(EntityCheckerManagerImpl.java:64)
org.aksw.gerbil.dataset.AbstractDatasetConfiguration.getPreparedDataset(AbstractDatasetConfiguration.java:79)
org.aksw.gerbil.dataset.SingletonDatasetConfigImpl.getPreparedDataset(SingletonDatasetConfigImpl.java:50)
org.aksw.gerbil.dataset.AbstractDatasetConfiguration.getDataset(AbstractDatasetConfiguration.java:50)
org.aksw.gerbil.execute.ExperimentTask.run(ExperimentTask.java:106)
org.aksw.simba.topicmodeling.concurrent.workers.WorkerImpl.run(WorkerImpl.java:44)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
java.base/java.lang.Thread.run(Thread.java:1570)

additionally that appears in my terminal when I try to run it on the MSNBC dataset.
I added the annotater using the config web UI and it worked but I can just not run the model entirely. It specifically only works with KORE50.

Screenshot 2024-10-10 at 15 00 20

@MichaelRoeder
Copy link
Member

From the information under /running, we can see that GERBIL is investing time to check the dataset for faulty entities. This is another step after the sameAs retrieval that can (unfortunately) take quite a lot of time. If you would like to skip this step, please have a look at the wiki article about HTTP-based entity checking.

I am not sure whether I understand the Python stack trace of your program correctly. It seems like GERBIL is sending a request without a NIF context. However, that would be very surprising. 🤔
Maybe you can turn off the entity checking and then try the different datasets that you would like to use. If the error still occurs, I am happy to have another look at it 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants