Add method to get URL Status (returns an URLItem)

Implemented only for MemoryFrontier and RocksDb Unfortunately the internal storage doesn't make a distinction between Discovered and Known URLs which have to be refetched So all scheduled items will be returned as ill always return KwownURLItem or Status.NOT_FOUND runtime exception Signed-off-by: Laurent Klock <[email protected]>
crawler-commons · Aug 8, 2024 · c242389 · c242389
1 parent 3ec6f44
commit c242389
Show file tree

Hide file tree

Showing 16 changed files with 3,364 additions and 873 deletions.
diff --git a/API/pom.xml b/API/pom.xml
@@ -15,8 +15,8 @@
 	<description>URLFrontier Java API</description>
 
 	<properties>
-		<grpc.version>1.53.0</grpc.version>
-		<protoc.version>3.20.3</protoc.version>
+		<grpc.version>1.65.1</grpc.version>
+		<protoc.version>3.25.4</protoc.version>
 	</properties>
 
 	<dependencies>

diff --git a/API/src/main/java/crawlercommons/urlfrontier/URLFrontierGrpc.java b/API/src/main/java/crawlercommons/urlfrontier/URLFrontierGrpc.java