Good Morning!
I will try by my side, I will keep you updated!
Regards!
Miguel.
Hi!
It is working perfectly for me (attached screenshots).
I checked your properties and there is just one difference: you are using a different name for NRTM database (I am using NRTM_LOCAL and you NRTM) as far as your database name is “NRTM” you shouldn’t have any problems.
My databases are (ACL_LOCAL, INTERNALS_LOCAL, LOCAL, MAILUPDATES_LOCAL, NRTM_LOCAL, WHOIS_LOCAL, WHOIS_MIRROR_RIPE_GRS, common databases (for schemas info), test databases)
The schema of LOCAL and WHOIS_MIRROR_RIPE_GRS should be the same, and the data should be the same as well.
Could you check your databases (probably a problem with WHOIS_MIRROR_RIPE_GRS?)
Hi Miguel,
Thank you very much for your kind attention and help!
so far i made 2 attemts to launch the whois server,
for now (second attempt) i have these databases (strange thing, my output for the max(serial_id) differs gravely from yours):
but on both attempts i have the same failure, after issuing:
root@RIPE:/home/dbase/whois# mysqldump -udbint -p LOCAL > LOCAL.sql
Enter password:
root@RIPE:/home/dbase/whois#
root@RIPE:/home/dbase/whois# mysql -udbint -p WHOIS_MIRROR_RIPE_GRS < LOCAL.sql
Enter password:
root@RIPE:/home/dbase/whois#
i have these commands succesfully completed, the test (telnet 127.0.0.1 then 193.0.0.1 - 193.0.7.255) works, but - the whois server itself tends to stop, here are the logs:
Exception in thread “main” org.jline.reader.UserInterruptException
at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:583)
at org.jline.reader.impl.LineReaderImpl.readLine(LineReaderImpl.java:390)
at org.cyclopsgroup.jmxterm.io.JlineCommandInput.readLine(JlineCommandInput.java:35)
at org.cyclopsgroup.jmxterm.boot.CliMain.execute(CliMain.java:147)
at org.cyclopsgroup.jmxterm.boot.CliMain.main(CliMain.java:41)
2023-07-24T22:12:36,392 WARN [ElasticFullTextIndex] Elasticsearch is not enabled
2023-07-24T22:12:45,383 INFO [RpslObjectsExporter] Exported 500000 objects
Killed
root@RIPE:/home/dbase/whois#
or:
2023-07-25T08:44:58,377 ERROR [ElasticIndexService] Elasticsearch cluster is not running
2023-07-25T08:44:58,377 WARN [ElasticFullTextIndex] Elasticsearch is not enabled
2023-07-25T08:45:58,786 ERROR [ElasticIndexService] Elasticsearch cluster is not running
2023-07-25T08:45:58,792 WARN [ElasticFullTextIndex] Elasticsearch is not enabled
2023-07-25T08:46:58,996 ERROR [ElasticIndexService] Elasticsearch cluster is not running
2023-07-25T08:46:58,996 WARN [ElasticFullTextIndex] Elasticsearch is not enabled
2023-07-25T08:47:08,629 INFO [ScheduledTaskAspect] Starting scheduled task: net.ripe.db.whois.scheduler.task.autnum.LegacyAutnumReloadTask@ca31ad7
2023-07-25T08:47:08,631 INFO [ScheduledTaskAspect] Scheduled task: net.ripe.db.whois.scheduler.task.autnum.LegacyAutnumReloadTask@ca31ad7 took 1.253 ms
2023-07-25T08:47:13,483 INFO [WhoisLog] 1776199073 PW-QRY-INFO <1+3+0> 273ms [127.0.0.1] – 193.0.0.1 - 193.0.7.255
2023-07-25T08:47:59,018 ERROR [ElasticIndexService] Elasticsearch cluster is not running
2023-07-25T08:47:59,019 WARN [ElasticFullTextIndex] Elasticsearch is not enabled
2023-07-25T08:48:47,097 INFO [WhoisServer] Preparing to shut down net.ripe.db.whois.nrtm.client.NrtmImporter@2acca224
2023-07-25T08:48:47,097 INFO [WhoisServer] Preparing to shut down net.ripe.db.whois.nrtm.NrtmServer@7fb1820d
2023-07-25T08:48:47,185 INFO [WhoisServer] Preparing to shut down net.ripe.db.whois.query.QueryServer@22decfa2
2023-07-25T08:48:47,238 INFO [WhoisServer] Preparing to shut down net.ripe.db.whois.api.httpserver.JettyBootstrap@1cc5d8a9
2023-07-25T08:48:47,279 INFO [Server] Stopped Server@1435b17{STOPPING}[10.0.14,sto=0]
2023-07-25T08:48:47,310 INFO [AbstractConnector] Stopped ServerConnector@2f803d8{HTTP/1.1, (http/1.1)}{0.0.0.0:1080}
2023-07-25T08:48:47,350 INFO [WhoisDoSFilter] DoSFilter IP whitelist cleared
2023-07-25T08:48:47,557 INFO [ContextHandler] Stopped o.e.j.w.WebAppContext@1fd1deed{/,file:///home/dbase/whois/src/main/webapp,STOPPED}
2023-07-25T08:48:47,592 INFO [WhoisServer] Preparing to shut down net.ripe.db.whois.api.mail.dequeue.MessageDequeue@8947a4b
2023-07-25T08:48:47,592 INFO [MessageDequeue] Message dequeue stopping
2023-07-25T08:48:47,593 INFO [MessageDequeue] Message dequeue stopped
2023-07-25T08:48:47,594 INFO [WhoisServer] Stopped net.ripe.db.whois.nrtm.client.NrtmImporter@2acca224 in 22.05 ?s
2023-07-25T08:48:47,594 INFO [NrtmServer] Shutting down
2023-07-25T08:48:47,594 INFO [NrtmChannelsRegistry] Closing 0 open channels.
2023-07-25T08:48:47,613 INFO [WhoisServer] Stopped net.ripe.db.whois.nrtm.NrtmServer@7fb1820d in 19.67 ms
2023-07-25T08:48:47,617 INFO [NrtmChannelsRegistry] Closed all channels.
2023-07-25T08:48:47,616 INFO [WhoisServer] Stopped net.ripe.db.whois.query.QueryServer@22decfa2 in 16.32 ?s
2023-07-25T08:48:47,621 INFO [WhoisServer] Stopped net.ripe.db.whois.api.httpserver.JettyBootstrap@1cc5d8a9 in 480.8 ?s
2023-07-25T08:48:47,621 INFO [MessageDequeue] Message dequeue stopping
2023-07-25T08:48:47,621 INFO [MessageDequeue] Message dequeue stopped
2023-07-25T08:48:47,621 INFO [WhoisServer] Stopped net.ripe.db.whois.api.mail.dequeue.MessageDequeue@8947a4b in 17.59 ?s
2023-07-25T08:48:47,706 INFO [ReadinessHealthCheck] Marked service as not ready to receive traffic
2023-07-25T08:48:47,706 INFO [WhoisServer] waiting for 0 seconds before starting to close spring context
2023-07-25T08:48:47,708 INFO [WhoisServer] starting to destroy beans
2023-07-25T08:48:48,582 INFO [WhoisServer] Whois server stopped in 1.485 s
root@RIPE:/home/dbase/whois#
before the “mysql -udbint -p WHOIS_MIRROR_RIPE_GRS < LOCAL.sql” i experience no such issue, but after it the server tends to stop after 1 hour, then 10 mins then immediately…
So, in total, i have:
- until the “mysql -udbint -p WHOIS_MIRROR_RIPE_GRS < LOCAL.sql” everything went ok (i noted no issues), but after it the whois server starts to behave unstable, so that finally i even can’t start it;
- the initial RIPE dump being uploaded for 4 days, IDK if that may affect;
may be it is related to my machine, as it is weak enough, may be smt else…
I think i will give it another try from the scratch, it would be great if you may continue to support me here, to try to finalyze the installation
Hi!
First of all you need to stop the server, then run
mysqldump -udbint -p LOCAL > LOCAL.sql
AND
mysql -udbint -p WHOIS_MIRROR_RIPE_GRS < LOCAL.sql
And finally start the server again, you can not restore the dump while the server is on.
Here is specifying to kill before the dump.
The serial ID probably is different because you didn’t get to this point:
UPDATE serials SET serial_id = RIPE.CURRENTSERIAL WHERE serial_id = max_serial_id;
ALTER TABLE serials AUTO_INCREMENT = RIPE.CURRENTSERIAL;
I hope that this fixes your problem.
Hi Miguel!,
as for:
i stopped/started it for sure, just did not put the sysctl part in my snippet…
and here:
yep, this time i didn’t reached it, but previously i did, and the numbers were inlined,
but in both cases my problem is system crash after uploading
i’ll try once more (as it already runs), then i hope to get a suitable machine to try it again.
Hi Miguel,
please, i noted an error in the logs:
2023-08-02T23:17:54,189 WARN [IpResourceConfiguration] Refresh failed due to org.springframework.jdbc.BadSqlGrammarException: StatementCallback; bad SQL grammar [SELECT prefix FROM acl_denied]; nested exception is java.sql.SQLSyntaxErrorException: (conn=33) Table ‘ACL_LOCAL.acl_denied’ doesn’t exist
can it probably cause problems during/after dump load?
Hi!
You should have that table if you have followed the previous steps
You need to create ACL_LOCAL database and launch acl_schema.sql, the table ‘all_denied’ is the first one there. Would you mind trying to load the script one more time, and running the SELECT manually.
Well, strange, but the table is in the list:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 125
Server version: 10.6.12-MariaDB-0ubuntu0.22.04.1 Ubuntu 22.04Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.
MariaDB [(none)]> use ACL_LOCAL;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -ADatabase changed
MariaDB [ACL_LOCAL]> show tables;
±--------------------+
| Tables_in_ACL_LOCAL |
±--------------------+
| acl_denied |
| acl_event |
| acl_limit |
| acl_proxy |
| apikeys |
| override_users |
| version |
±--------------------+
7 rows in set (0,000 sec)MariaDB [ACL_LOCAL]>
is it ok to remove/add the database during the dump upload? Assume not of course…
Or interrupt the upload and start from the beginning
and if possible, another error:
2023-08-03T07:05:21,546 INFO [DatabaseHealthCheck] Database connection failed health check: Incorrect result size: expected 1, actual 0
2023-08-03T08:10:26,556 ERROR [IpTreeUpdater] Unexpected
java.lang.IllegalArgumentException: No cached ipTree for source: LOCAL
at net.ripe.db.whois.common.iptree.IpTreeCacheManager.update(IpTreeCacheManager.java:210) ~[whois.jar:?]
at net.ripe.db.whois.common.iptree.IpTreeCacheManager.update(IpTreeCacheManager.java:199) ~[whois.jar:?]
at net.ripe.db.whois.common.iptree.IpTreeUpdater$3.execute(IpTreeUpdater.java:124) ~[whois.jar:?]
at net.ripe.db.whois.common.iptree.IpTreeUpdater$4.run(IpTreeUpdater.java:140) [whois.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:833) [?:?]
is it possible for me to check which database fails?
i have:
MariaDB [ACL_LOCAL]> show databases;
±----------------------+
| Database |
±----------------------+
| ACL_LOCAL |
| INTERNALS_LOCAL |
| LOCAL |
| MAILUPDATES_LOCAL |
| NRTM |
| NRTM_LOCAL |
| WHOIS_LOCAL |
| WHOIS_MIRROR_RIPE_GRS |
| dnscheck_local |
| information_schema |
| mysql |
| performance_schema |
| sys |
±----------------------+
13 rows in set (0,011 sec)MariaDB [ACL_LOCAL]>
Hi!
I have never tried to delete a database when a server is running (I always stop the server that is using the database before changing the database). However, it makes sense that the healthcheck complains if you drop a database, or have some problems with an open transaction.
In this case, you can’t check which database is failing.
I would recommend waiting until you have the new machine, or try the process using free command to have some insight into the memory that you are using.
I have the same databases.
1 Like
Hi Miguel,
another snag, during the dump upload i got an error:
$>run loadDump initialimport ripe.db.gz
#calling operation loadDump of mbean net.ripe.db.whois:name=Bootstrap with params [initialimport, ripe.db.gz]
#operation returns:
Error in pass 2 in ‘[inetnum] 195.80.183.0 - 195.80.183.31’: org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [INSERT INTO history SELECT object_id, sequence_id, timestamp, object_type, object, pkey FROM last WHERE object_id = ? and sequence_id = ?]; SQL state [null]; error code [0]; An SQLException was provoked by the following failure: java.lang.reflect.UndeclaredThrowableException; nested exception is java.sql.SQLException: An SQLException was provoked by the following failure: java.lang.reflect.UndeclaredThrowableException
may it be an issue with the dump object (at first glance it’s quite ok) or it rather smt with the processing ? Yet, i restarted from 12 hours old snapshot.
Meanwhile soon i get the dedicated server, Monday-Tueseday.
Hi!
I tried by my side, on my own machine and have no errors.
I think that these errors are happening because of multiple tries and unstable left overs from previous tries.
I recommend waiting for the new server and try from scratch.
Let me know how it goes!
1 Like
Hi Miguel, good morning!,
i got the laptop, yesterday, launched server and upload and it’s done in 8 hours.
Now going to proceed with NRTM
Besides, during the upload there was an error:
Scheduled task: net.ripe.db.whois.scheduler.task.loader.Bootstrap@5927473a took 222.0 μs
2023-08-07T22:00:00,007 INFO [ScheduledTaskAspect] Starting scheduled task: net.ripe.db.whois.scheduler.task.grs.GrsImporter@6d9bcf19
2023-08-07T22:00:00,007 INFO [GrsImporter] GRS import is not enabled
2023-08-07T22:00:00,007 INFO [ScheduledTaskAspect] Scheduled task: net.ripe.db.whois.scheduler.task.grs.GrsImporter@6d9bcf19 took 116.3 μs
2023-08-07T22:00:00,008 INFO [ScheduledTaskAspect] Starting scheduled task: net.ripe.db.whois.scheduler.task.export.DatabaseTextExport@295e989
2023-08-07T22:00:00,008 INFO [RpslObjectsExporter] Database export started
2023-08-07T22:00:00,008 INFO [RpslObjectsExporter] Use existing: var/export
2023-08-07T22:00:00,008 INFO [RpslObjectsExporter] Delete existing: var/export_tmp
2023-08-07T22:00:00,008 INFO [RpslObjectsExporter] Created: var/export_tmp
2023-08-07T22:00:00,363 ERROR [TaskUtils$LoggingErrorHandler] Unexpected error occurred in scheduled task
java.lang.NullPointerException: Cannot invoke “java.lang.Integer.intValue()” because the return value of “org.springframework.jdbc.core.JdbcTemplate.queryForObject(String, java.lang.Class)” is null
at net.ripe.db.whois.scheduler.task.export.dao.JdbcExportDao.getMaxSerial(JdbcExportDao.java:30) ~[whois.jar:?]
at net.ripe.db.whois.scheduler.task.export.RpslObjectsExporter.exportToFiles_aroundBody0(RpslObjectsExporter.java:97) ~[whois.jar:?]
at net.ripe.db.whois.scheduler.task.export.RpslObjectsExporter$AjcClosure1.run(RpslObjectsExporter.java:1) ~[whois.jar:?]
but i can test this:
telnet localhost 1043
193.0.0.1 - 193.0.7.255
Hi!
That’s fantastic news!
Let me track that error. This error shouldn’t stuck you, It seems that some attribute of some object is not well formatted, you can go on.
Let me know how the NRTM setup goes!
1 Like
Seems, NRTM works:
But search API not:
GET http://192.168.0.23:1080/search.json?query-string=195.25.33.160%20-%20195.25.33.175&flags=no-filtering&resource-holder=true 404 25 ms
Accept: application/json
User-Agent: PostmanRuntime/7.32.3
Cache-Control: no-cache
Postman-Token: 0ce2e1b6-b9b3-4a12-8f82-4661ba2513c4
Host: 192.168.0.23:1080
Accept-Encoding: gzip, deflate, br
Connection: keep-aliveHTTP/1.1 404 Not Found
Cache-Control: must-revalidate,no-cache,no-store
Content-Type: application/json
Content-Length: 84
Server: Jetty(10.0.15){
“servlet”:“default”,
“message”:“Not Found”,
“url”:“/search.json”,
“status”:“404”
}
Hi!
Don’t miss whois!!
Just tested in my local.
Regards!
Hi Miguel!,
it works perfectly, thank you very much!!!
Glad to hear that.
My pleasure
Please do not hesitate to contact us if you have further questions.
Hi Miguel,
First of all, thank you ever so much for your kind help and especially, your patience!
And, let me please reassure you, that after some tests and playing around with the new toy, “i’ll be back” with a new portion of silly annoying questions, just give me several days please