Replies: 3 comments 2 replies
-
|
My thoughts from the meeting today: I agree that the proxy approach will be less churn for developers and if the proxy can integrate with the internal compliance tools automatically, then that will be a big win. The syncing approach would provide the added benefit of internal code mirroring of any upstream project, which could also mean internal ci/cd could be integrated with these mirrors before the commit in questions is pushed upstream, but this would be extra effort and relies on internal mirroring infrastructure to be present and also scalable. P.S thanks for a great introduction today |
Beta Was this translation helpful? Give feedback.
-
|
Great feedback @adamj-codethink , thank you; it's been great to have you on the call yesterday, I hope you'll join next meetings too! Re. your comments on syncing, I'd like to reflect some of these thoughts in the content above, so I'll try to dig a bit deeper on your comments.....
Just to be clear, you're referring to the fact that you're not affected by a central infrastructure downtime? I'd also consider the counter argument: any OSS contribution coming from your firm depends on the uptime of your internal Git system, and if it's down, you're not going to be able to do anything. I think it would take quite some DevOps work to come up with a Git infrastructure that provides less downtimes than GitHub (given that is their core business). That said, it's a choice that every FI must take by considering their own environment, rules and regulations, so it's crucial that we describe the challenge as-is and let others take their conclusions.
The CI/CD "chain" is again something that would be fully redundant with the operations happening in the public and mirrored repos, leading to some downsides, for example the time for changes to hit the final repo would double, same as costs to pay the CI/CD infrastructure. When working with microservices and containerised architectures, it's quite common to heavily rely on integration tests to check version compatibility across modules; these workloads are heavy, long and expensive; I would try as much as possible to define them once, instead of replicating them across different infrastructures. Finally, there's another difference between proxying and syncing which is IMO worth mentioning: using a syncing mechanism, every project must be setup internally first, before allowing developers to work; and it would be quite hard to provide a self-service approach for internal developers. The proxying mechanism COULD be used self-service, assuming there's a way to enforce the proxy configuration at workstation or network level. |
Beta Was this translation helpful? Give feedback.
-
|
MVP Requirements
|
Beta Was this translation helpful? Give feedback.
-
|
Adding a couple suggestions do the MVP requirements:
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks @junjikatto , I've added these items into finos/community#65 as well |
Beta Was this translation helpful? Give feedback.
-
This document hosts a discussion - started on ODP issue #155 - that aims to give guidance to Financial Institutions that want to setup infrastructure and processes in place that allow firm employees to contribute to existing Open Source projects.
The outcome of this initiative should include the following assets:
Strategies
Proxy
Setting up a proxy that org developers must use in order to reach (HTTPS and SSH) endpoints on github.com ; the proxy rejects or approves Git operations, based on checks and validations in place to comply with corporate regulations and bylaws.
The Git Proxy is a proof of concept developed by @grovesy that intends to deliver this approach: it proxies the push and merge Git operations and runs them through a chain of validators, which have access to the old and new content of the change. The validator can implement any logic, but some of the examples that have been mentioned so far are:
Firewall Rules
Similar to the proxy, but identifies checks and validations at firewall level; this solution doesn't seem to allow to implement (manual) compliance code review workflows, as it doesn't allow to store and forward Git operations (needs more investigation).
Syncing
Setting up an internal Git server (behind corporate firewall) and define automation to sync branches across the corporate Git server and github.com/finos/* ; when branches are public, authors must submit a PR against
masterand the project maintainers will review (and possibly approve) it; Git mirroring can also be used as a mechanism to sync, and several (if not all) Git Enterprise products provide a way to setup mirroring and automation around it.The syncing mechanism has the ability to replicate contents and processes (ie CI/CD) internally, leading to more independence when operating internally, but on the other hand....
Which strategy to choose
Choosing one or the other strategy is something that heavily depends on the organization setup, org chart, identity management, security and compliance processes and much more.
The majority of FINOS members are using an internal Git server, given that most of their development activity is internal, however, we see many of them facing technical challenges trying to adapt this strategy in order to contribute to open source.
Few FINOS members use the proxy strategy to contribute to some OSS projects (while keeping an internal Git server for internal developments), which delivers a more dev-ergonomics workflow.
FAQ
How to control who can contribute to specific projects on public repos? Should we create proxy rules specific to access white listed repo URLs? As of today, no developer has access to public repos.
The proxy approach allows to easily implement repo whitelisting. By integrating the proxy with the internal's firm IdM, it's quite trivial to extend whitelisting to Git repositories and authors.
The sync approach can allows to control access on the internal Git server (easily configurable with an internal IdM like LDAP).
What steps should we take before whitelist a repo to be added to the white list?
It would be good to have a general assessment of the project, below is top 5 items that we deliver in our project (documentation) template.
Other than that, unless there's a question whether an activity on this repository would hurt the firm, I don't see any reason to hold off.
Which controls should we implement in order to allow contribution directly from the developer's machine?
Implementing controls on developer's machine is a risky approach, as it limits the scope of controls only when the developer works from that specific workstation; for this reason, controls tend to be implemented server-side.
How to implement a simple yet effective compliance control? Should we use a git proxy whenever there's a pull request being sent to the public repo? Should we start with manual validation for IP and Data Leakage?
The answer to these questions passes through the understanding of the 2 approaches we introduced above, proxy or syncing. Proxy definitely seems to be the most dev-ergonomics workflow.
The Git syncing approach requires a specific syncing configuration for each repository/team, in order to connect the internal with the public repo. Below is a development workflow that can be configured using this strategy:
finos_prefix)finos_branches into thegithub.com/finos/<repository name>.gitendpoint; it also pulls changes from the finos upstreammasterbranchfinos_feature branch is available, the author can submit a Pull Request against themasterbranch.Starting with manual validations seems like a very good way to start working on real scenarios and validate requirements.
Should we create an internal repo mirroring the public, just for the purpose of contributing?
I see mirroring as a way to implement the syncing approach; I cannot recall any member or other contributor that have used it. The mirroring feature is normally provided by Enterprise Git Servers deployed behind corporate firewalls, so it depends on the product if/how they support this feature and how to address Identity Management (mapping internal Git users with GitHub usernames) and controls (IP, compliance, security, quality)
What are the best practices to create a corporate github account for contribution before we allow developers to associate their github account to our corporate account?
Haven't seen many best practices around creating corporate GitHub accounts (/CC @jonesbr), but I'd suggest to request:
Beta Was this translation helpful? Give feedback.
All reactions