Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PMM-11341 Auto discovery limit. #2568

Merged
merged 62 commits into from
Nov 16, 2023
Merged
Show file tree
Hide file tree
Changes from 52 commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
55d5655
PMM-11341 Auto discovery control.
JiriCtvrtka Oct 26, 2023
2069925
PMM-11341 Fix.
JiriCtvrtka Oct 26, 2023
7fcbd3b
PMM-11341 Same comments on all places.
JiriCtvrtka Oct 26, 2023
1f4fc9f
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Oct 26, 2023
5009442
PMM-11341 Remove live service info broker scrape.
JiriCtvrtka Oct 26, 2023
07452fd
PMM-11341 Lint.
JiriCtvrtka Oct 26, 2023
a00925b
PMM-11341 Connection checker for PG.
JiriCtvrtka Oct 26, 2023
a170704
PMM-11341 Fix comment.
JiriCtvrtka Oct 30, 2023
cec000c
PMM-11341 Discover limit field even for QAN agents.
JiriCtvrtka Oct 30, 2023
0066b0b
PMM-11341 Remove version check from connection checker.
JiriCtvrtka Oct 30, 2023
884c5e5
PMM-11341 Remove whole PG connection check block on managed side.
JiriCtvrtka Oct 30, 2023
62dc4cc
Revert "PMM-11341 Discover limit field even for QAN agents."
JiriCtvrtka Oct 30, 2023
73eedc7
PMM-11341 Remove now unused field.
JiriCtvrtka Oct 30, 2023
0b865c4
PMM-11341 Fix bad proto numbering after revert.
JiriCtvrtka Oct 30, 2023
5e1f6dc
PMM-11341 Agreed changes.
JiriCtvrtka Oct 30, 2023
d0970c7
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Oct 30, 2023
cdadc1c
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Oct 30, 2023
c7ac422
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Oct 31, 2023
3870ab1
PMM-11341 agent, admin identify as 2.41.0 version.
JiriCtvrtka Oct 31, 2023
b8c95c9
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 1, 2023
b5e94c2
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 1, 2023
e1435a3
PMM-11341 Add check for auto discovery limit value.
JiriCtvrtka Nov 1, 2023
bf12226
Merge branch 'PMM-11341-autodiscovery-control' into PMM-11341-autodis…
JiriCtvrtka Nov 1, 2023
8969902
PMM-11341 Count without excluded DBs.
JiriCtvrtka Nov 1, 2023
2ace2a6
PMM-11341 Fix to scan.
JiriCtvrtka Nov 1, 2023
38d23fa
Merge branch 'PMM-11341-autodiscovery-control-241' into PMM-11341-aut…
JiriCtvrtka Nov 1, 2023
bc49447
PMM-11341 Remove forced version.
JiriCtvrtka Nov 1, 2023
da4b10e
PMM-11341 Gen.
JiriCtvrtka Nov 1, 2023
67ca78e
PMM-11341 Bad naming.
JiriCtvrtka Nov 1, 2023
87947d7
PMM-11341 Close to rows.
JiriCtvrtka Nov 1, 2023
6f99bb1
PMM-11341 Fix.
JiriCtvrtka Nov 1, 2023
67a9bf5
PMM-11341 Status error instead common one.
JiriCtvrtka Nov 1, 2023
d650a2b
PMM-11341 Naming.
JiriCtvrtka Nov 6, 2023
b051579
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 6, 2023
3df1a9b
PMM-11341 Fix typo.
JiriCtvrtka Nov 7, 2023
b3cc48b
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 7, 2023
52ef2d0
PMM-11341 RDS field.
JiriCtvrtka Nov 8, 2023
45f4426
PMM-11341 RDS.
JiriCtvrtka Nov 8, 2023
df7b822
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 9, 2023
01ab5c9
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 9, 2023
85656be
PMM-11341Align with table stats limit.
JiriCtvrtka Nov 9, 2023
766547c
PMM-11341 Align with MySQL.
JiriCtvrtka Nov 9, 2023
f9fc380
PMM-11341 Format.
JiriCtvrtka Nov 9, 2023
2a9830a
PMM-11341 Lint.
JiriCtvrtka Nov 9, 2023
76c2288
PMM-11341 Lower case.
JiriCtvrtka Nov 9, 2023
35bf48c
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 9, 2023
b0a907c
PMM-11341 FIx MySQL, PG pmm-admin description.
JiriCtvrtka Nov 9, 2023
4935cab
PMM-11341 DB list query moved above.
JiriCtvrtka Nov 9, 2023
6002c41
PMM-11341 Lint.
JiriCtvrtka Nov 9, 2023
eda8268
PMM-11341 MySQL, PG comments fix for limit.
JiriCtvrtka Nov 9, 2023
8eb1175
PMM-11341 Lint.
JiriCtvrtka Nov 9, 2023
dbfcc1b
PMM-11341 Remove nolint.
JiriCtvrtka Nov 9, 2023
0de232a
PMM-11341 Ifs refactor.
JiriCtvrtka Nov 9, 2023
6d18ef9
PMM-12341 Refactor.
JiriCtvrtka Nov 9, 2023
92bfc60
PMM-11341 Change default limit to 10.
JiriCtvrtka Nov 13, 2023
6a2d55b
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 13, 2023
dcc83e1
PMM-11341 Small description change.
JiriCtvrtka Nov 14, 2023
6235b38
PMM-11341 Fix RDS test after change of default value.
JiriCtvrtka Nov 14, 2023
efc4350
PMM-11341 Fix description for MySQL.
JiriCtvrtka Nov 14, 2023
c045ca7
Merge branch 'main' into PMM-11341-autodiscovery-control
JiriCtvrtka Nov 16, 2023
630cf5f
PMM-11341 Fix mariaDB 10.4 expected queryID.
JiriCtvrtka Nov 16, 2023
221ab29
PMM-11341 Fix API tests for add PG.
JiriCtvrtka Nov 16, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions admin/commands/inventory/add_agent_mysqld_exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ func (res *addAgentMysqldExporterResult) TablestatStatus() string {
}

switch {
case res.Agent.TablestatsGroupTableLimit == 0: // no limit
case res.Agent.TablestatsGroupTableLimit == 0: // server defined
s += " (the table count limit is not set)."
case res.Agent.TablestatsGroupTableLimit < 0: // always disabled
s += " (always)."
Expand Down Expand Up @@ -94,7 +94,7 @@ type AddAgentMysqldExporterCommand struct {
TLSCAFile string `name:"tls-ca" help:"Path to certificate authority certificate file"`
TLSCertFile string `name:"tls-cert" help:"Path to client certificate file"`
TLSKeyFile string `name:"tls-key" help:"Path to client key file"`
TablestatsGroupTableLimit int32 `placeholder:"number" help:"Tablestats group collectors will be disabled if there are more than that number of tables (default: 0 - always enabled; negative value - always disabled)"`
TablestatsGroupTableLimit int32 `placeholder:"number" help:"Tablestats group collectors will be disabled if there are more than that number of tables (default: server-defined)"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wow, what about negative value? was it disabling or not?

PushMetrics bool `help:"Enables push metrics model flow, it will be sent to the server by an agent"`
DisableCollectors []string `help:"Comma-separated list of collector names to exclude from exporter"`
LogLevel string `enum:"debug,info,warn,error" default:"warn" help:"Service logging level. One of: [debug, info, warn, error]"`
Expand Down
4 changes: 4 additions & 0 deletions admin/commands/inventory/add_agent_postgres_exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ func (res *addAgentPostgresExporterResult) String() string {
}

// AddAgentPostgresExporterCommand is used by Kong for CLI flags and commands.
//
//nolint:lll
type AddAgentPostgresExporterCommand struct {
PMMAgentID string `arg:"" help:"The pmm-agent identifier which runs this instance"`
ServiceID string `arg:"" help:"Service identifier"`
Expand All @@ -62,6 +64,7 @@ type AddAgentPostgresExporterCommand struct {
TLSCertFile string `help:"TLS certificate file"`
TLSKeyFile string `help:"TLS certificate key file"`
LogLevel string `enum:"debug,info,warn,error" default:"warn" help:"Service logging level. One of: [debug, info, warn, error]"`
AutoDiscoveryLimit int32 `default:"0" placeholder:"NUMBER" help:"Auto-discovery will be disabled if there are more than that number of databases (default: server-defined)"`
}

func (cmd *AddAgentPostgresExporterCommand) RunCmd() (commands.Result, error) {
Expand Down Expand Up @@ -99,6 +102,7 @@ func (cmd *AddAgentPostgresExporterCommand) RunCmd() (commands.Result, error) {
SkipConnectionCheck: cmd.SkipConnectionCheck,
PushMetrics: cmd.PushMetrics,
DisableCollectors: commands.ParseDisableCollectors(cmd.DisableCollectors),
AutoDiscoveryLimit: cmd.AutoDiscoveryLimit,

TLS: cmd.TLS,
TLSSkipVerify: cmd.TLSSkipVerify,
Expand Down
2 changes: 1 addition & 1 deletion admin/commands/management/add_mysql.go
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ func (res *addMySQLResult) TablestatStatus() string {
s := "Table statistics collection " + status

switch {
case res.MysqldExporter.TablestatsGroupTableLimit == 0: // no limit
case res.MysqldExporter.TablestatsGroupTableLimit == 0: // server defined
s += " (the table count limit is not set)."
case res.MysqldExporter.TablestatsGroupTableLimit < 0: // always disabled
s += " (always)."
Expand Down
2 changes: 2 additions & 0 deletions admin/commands/management/add_postgresql.go
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ type AddPostgreSQLCommand struct {
DisableQueryExamples bool `name:"disable-queryexamples" help:"Disable collection of query examples"`
MetricsMode string `enum:"${metricsModesEnum}" default:"auto" help:"Metrics flow mode, can be push - agent will push metrics, pull - server scrape metrics from agent or auto - chosen by server"`
DisableCollectors []string `help:"Comma-separated list of collector names to exclude from exporter"`
AutoDiscoveryLimit int32 `default:"0" placeholder:"NUMBER" help:"Auto-discovery will be disabled if there are more than that number of databases (default: server-defined)"`

AddCommonFlags
AddLogLevelNoFatalFlags
Expand Down Expand Up @@ -203,6 +204,7 @@ func (cmd *AddPostgreSQLCommand) RunCmd() (commands.Result, error) {
DisableQueryExamples: cmd.DisableQueryExamples,
MetricsMode: pointer.ToString(strings.ToUpper(cmd.MetricsMode)),
DisableCollectors: commands.ParseDisableCollectors(cmd.DisableCollectors),
AutoDiscoveryLimit: cmd.AutoDiscoveryLimit,
LogLevel: &cmd.AddLogLevel,
},
Context: commands.Ctx,
Expand Down
22 changes: 21 additions & 1 deletion agent/serviceinfobroker/service_info_broker.go
Original file line number Diff line number Diff line change
Expand Up @@ -219,12 +219,32 @@ func (sib *ServiceInfoBroker) getPostgreSQLInfo(ctx context.Context, dsn string,
db := sql.OpenDB(c)
defer db.Close() //nolint:errcheck

var databaseList []string
databaseListQuery := "SELECT /* agent='serviceinfobroker' */ datname FROM pg_database WHERE datallowconn = true AND datistemplate = false AND has_database_privilege(current_user, datname, 'connect')" //nolint:lll
rows, err := db.QueryContext(ctx, databaseListQuery)
if err != nil {
res.Error = err.Error()
return &res
}
defer rows.Close() //nolint:errcheck
for rows.Next() {
var databaseName string
err := rows.Scan(&databaseName)
if err != nil {
res.Error = err.Error()
return &res
}

databaseList = append(databaseList, databaseName)
}
res.DatabaseList = databaseList

var version string
if err = db.QueryRowContext(ctx, "SHOW /* agent='serviceinfobroker' */ SERVER_VERSION").Scan(&version); err != nil {
res.Error = err.Error()
}

res.Version = version

return &res
}

Expand Down
Loading
Loading