In this post we’re going to look at the new functionalities that have been added to Application Composer
In this post we’re going to look at the new functionalities that have been added to Application Composer by re:Invent 2023. After announcing the support of all CloudFormation resources earlier in the year, Application Composer now allows editing StepFunctions within the same user interface and – even cooler – announces the integration of an IDE plugin that allows developers to build serverless functions locally.
Application Composer as a serverless, rapid prototyping service adds additional capabilities to empower developers building serverless applications
Application Composer, that was originally announced last year at re:Invent 2022, has gotten a lot of major improvements thoughout 2023. As we are right at re:Invent 2023, its time to look back on which new capabilities have been added and how they influence building serverless applications using AppComposer.
Supporting all CloudFormation resouces
Already a few weeks ago the team announced that all over 1000 CloudFormation resources are now supported by AppComposer. This really gave a big update and make it simpler to build all kind of serverless applications. However, as this only alled AppComposer to expose the resources, this still requires the developer to know all required connections between the different resources. I personally would love to get more “supported” resources (just like L2 resources in CDK) to be made available as part of AppComposer. I would hope that this will be an additional functionality soon.
Integrating additional services
With the integration of the Stepfunctions Workflow Studio within the same interface, developers can now build and end to end application within composer before using the generated SAM or CDK templates to trigger the deployment. As a next step I think it would be great to also be able to define Eventbridge Rules & Pipes within the same interface.
Local development and IDE integration
AppComposer announced a Visual Studio Code integration that makes it possible to build and design serverless applications right from your IDE!
With this feature, you can visualize your serverless applications without being within your browser or the AWS console – start building, whereever you are and whenever you want!
I have not been able to try out this functionality yet, but especially the integration with sam connect that allows to also directly deploy the changes you made to your picture / template will make a big different in building applications using AppComposer.
Also I think we should not underestimate the possibility that this offers to vizualize existing CloudFormation templates through either the IDE plugin or the AWS Console. This will help to explain big and difficult already existing applications and empowers teams to have a fruitful conversation about changes they would like to implement in existing templates, as have a visualization makes the conversation easier.
What’s next for Application Composer? What are my wishes?
Already last year I have asked to integrate AppComposer in CodeCatalyst and I believe that this would be an awesome possibility to quickly start serverless projects. Application Composer today feels like a playground – to make the service more usable, it needs to have a “deployment” component that allows you to automate the lifecycle of your serverless application (including a full CI/CD pipeline).
I also last year asked for creation of CDK out of Application Composer – or even importing it – but instead of investing into that direction AWS recently announced the existance of the CDK Builder Tool – wouldn’t it be better to merge those initiatives together?
As already mentioned above, supporting additional “CDK-L2-like” patterns – or maybe the “Patterns” from serverlessland.com would be amazing – so users do not need to know how to set up IAM roles, connections between API Gateway and Lambda, … manually would make this a much more usable product!
What are your thoughts around the recent announcements of AppComposer? What are your experiences with it?
In this article I will try to re:Cap a few of the announcements at re:Invent 2023 but also share my personal experiences and learnings that covers what I think that should be shared with the world…!
What happens in Vegas…
…should not always stay in Las Vegas! This year’s re:Invent has been another great experience for me and it was amazing to meet AWS enthutiasts from all of the world. I’ve learned a tons of stuff, saw a bunch of cool sessions and also experienced to be part of a big family. All the friendships that have been build in the past few days, the shared knowledge and experiences that have been shared have a big influecnce on myself and shape me.
The technical aspects of re:Invent
This year the technical aspects of re:Invent existed but where not as important to me as they used to be in my previous attendances. Of course AWS hd a bunch of important announcements – some of them bigger, some smaller. Renato has them written up at InfoQ and the AWS Newsblog has them covered too. Luc, the winner of this years “Now, Go Build” award 2023 has created a web application that helps you to read all of them and not miss a single one.
For me, there are a few that stand out:
Amazon Q – your new fellow, builder companion that will make you more productive and more successful going forward
Of course, there where a bunch of other announcement, minor and smaller ones, but these are the ones that I have remembered and thus they are meaningful to me. Now let’s move over to the more important aspects of re:Invent!
The community aspects of re:Invent
re:Invent 2023 has been once more a gathering of the AWS Community in one place and it has brought a lot of us together to talk, laugh and align. Not everyone was able to join us due to different reasons – but I am sure that you have felt the power of the community throughtout the week by following us on the different social channels.
Being part of the AWS Heros
As I posted last year, going to re:Invent means meeting with friends and getting together. Being an AWS Hero, made it more intense than before: We feel community out of our heart and that’s what makes us strong. Wherever I was in Las Vegas, I saw a fellow Hero.
We all have super powers and our powers are different. One of my super-powers is connecting people – and I hope that I was able to show this in the last few days.
Others have other powers – a few of use were able to present one of their talks – Anahit with her spciality around MSK, Anurag around data patterns and Ran on Lambda Power Tools. Others are great listeners and others have the vision of how things need to or should look in a few years – it was great to see everyones powers in one place and I know that combining them we can incluence to make things better!
Thank you, Taylor and the rest of the team, for creating this group and bringing us together again!
Working with Builders, User Group leaders and others from the community
The AWS Community consists of so much more than the Heros. Thank you, dear Community Builders – lead my Jason and the team – for being an unbelievable source of power throughtout the week. Your entuthiasm, your great ideas and your dedication are what makes us stronger. I’ve been reading a lot of the posts from Builders around the globe that were not able to make it to Las Vegas and it is energizing to see that.
The User Group Leaders that we have world wide on the other side help to thrive the AWS Community across the whole yearand bring us together regularly — to learn, to play or to share knowledge. Thank you all, for helping us to shape where the community goes and for making the community successful. I was glad to be able to meet a lot of you and share my experiences as welll as listening to your experiences.
I had the great pleasure to get the whole team of core contributors of the Speakers Directory together and we were able to present our project as well as take a picture of all of use 🙂
We are going to continue our investment and will help user group leaders to find speakers through our tool!
Working with AWS employees
This year, I’ve joined the club of many other Heros that go to sessions where they can meet AWS service team members that they have worked with before 🙂
I attended a few CodeCatalyst sessions to meet the team that I’ve been working with for more than 12 months “live and in person” and loved to see the energy and innovation live on stage – but I also attended other sessions just to say HI to certain speakers.
Employees at AWS are smart and can often tell you the perspective of WHY something has been build and it’s great to know some more background of a new feature. Thank you all for spending time with me and sharing your thoughts and passion with me!
To those AWS employees in the community and DevRel team – another big THANKS for making the event unforgettable with all of your dedication and support – I love spending time with you and creating new ideas on how to make the AWS community stronger and more engaging than ever before!
A look ahead…
As I try to use my time on the flight to put my head around what I am taking away from the last few days and from re:Invent 2023, I’m still digesting, as many others, what we have all learned and heard.
A few key take-aways:
AWS doesn’t feel “secure” anymore to be a market leader
innovation at AWS is coming (Q), but it’s still early stages
AWS keeps listening to their customers (see the DB2 RDS announcement and the StepFunction HTTP Integration)
Community Sessions (COM or DEV track) are the ones to attend at re:invent, or sessions that are AWS + a customer (level 300/400)
What I’m considering to do in the next 3 months
First of all I’m planning to cover the CodeCatalyst announcements at my YouTube channel to explain the impact of the new features to interested enterprise customers.
Of course I will continue my engagement in the AWS Förderverein DACH – we are planning another AWS Community Day in Munich next year!
I also plan to continue my work with the the CodeCatalyst team to shape the product – please let me know if YOU have input on what thte next important steps are.
I would love to work with the AWS team to, for 2024 at re:Invent organize another pre:Invent Community Hike and to talk about the possibility of hosting a complete track at re:Invent where Community Members join forces with AWS employees. I listened to a session (Ran Isenberg and Heitor Lessa) and that was a very powerful message.
Last but not least, I would like to help community members to grow and shape their careers in Cloud – if you need help or have questions, do not hesitate to reach out for questions, I’m happy to help or to connect you with someone that can help!
Thank you for reading until this point, if you have any feedback, let me know!
In this post you will get a short introduction and my personal assessment on how the new Packages component in CodeCatalyst can help you to set up your complete SDLC in Amazon CodeCatalyst.
Only npm supported – a early launch to show what we can expect going forward and to get feedback from users?
With the initial launch Amazon CodeCatalyst allows to manage npm package repositories inside CodeCatalyst. When seing this for the first time I was a bit concerned that this decision – to initially launch with only npm support – will not help a lot, as I expect that organizations would need to store other type of artifacts (jar files, containers, python paackages, …). But I think that at least for Cloud Native Projects that use typescript this will solve the problem of storing artifacts and accessing them natively within CodeCatalyst. Let’s look at how they can be used today.
Using packages repositories in CodeCatalyst
Today you can set up package repositories in CodeCatalyst and connect to upstream (public or private) repositories. You can also change the sort order and the upstream order of the package repositories. The documentation covers the different possibilities to set up repositories very well. You can access the package repositories from your local machine by setting up the connection locally
npm config set registry https://packages.region.codecatalyst.aws/npm/space-name/proj-name/repo-name/
After setting up multiple repositories you can set the order of retrieval and usage. You can also set it up as a “pass-through” repository and allow access to public repositories.
Using packages in CodeCatalyst
Packages in CodeCatalyst are integrated with workflows. You can read or store packages with native actions or by setting up the npm repository manually.
You can also read and publish packages from other systems by using Personal Access Tokens (PAT). The documentation outlines currently supported client commands.
The CodeCatalyst workflows will by default use the set up repositories that you have set up within CodeCatalyst.
What I think we need next in packages
With the launch of the packages component in CodeCatalyst AWS has added a definately missing functionality in CodeCatalyst. Users are not able to store artifacts and share them within a space which makes it possible to create, re-use and deploy immutable artifacts natively within the tool.
This is a much needed functionality, but as I already mentioned, I believe that limiting this to npm packages only limits the use cases for it. I would have expected npm, python and docker support at launch and was a bit disappointed that this was not included.
I do also believe that the “packages” functionality should be better integrated and allow further configuration options, especially when loading/reading packages from upstream repositories – e.g. being able to limit to only certain licences or ensuring that packages included are still “supported” (and regularly updated) or security-scanned.
These type of functionalities would have made the new option “meaningful” and would have empowered developers to build better software too.
I can imagine that we will see python, java and dockersupport pretty soon due to these reasons and it would be great to also support “internal” repositories as “upstream” repositories.
How do YOU think that this new functionality will help you to adapt CodeCatalyst? Please drop me a message on socials or an E-Mail!
In this post you will learn how the new integration of the IAM Identity center with Amazon CodeCatalyst allows to use your own IdP as single source of truth of users for CodeCatalyst spaces. We will look at how you can connect a space to an IdP and on how this can be used to set up certain permissions based on your IdP roles.
Single Sign On (SSO) using IAM Identity Center empowers CodeCatalyst users to use their own IdP to provision user accounts in CodeCatalyst
With the new integration for Amazon CodeCatalyst into the AWS Identity Center organizations can now connect to their own existing IdP (AzureAD, Ping, Okta, …) going through the AWS Identity Center.
Benefits of the new integration
With this new integration users and organizations go to the Space settings in CodeCatalyst and choose to integrate with an existing instance of the IAM Identity Center.
This also connects potentialy to any other IdP that uses the IAM Itentity Center as a proxy. Administrators will only need to manage roles and permissions in on central place. The roles from the IdP can then be mapped to certain team roles in Amazon CodeCatalyst.
What I don’ t like about the new integration
There are two things that I didn’t like when looking at this integration:
Why do we need to go through the IAM Identity Center and cannot allow direct SAML or OIDC integration for a specific space? With the “need” to have an IAM Identity center organizations also need to directly connect to an existing AWS Account to manage connectivity – and this once again brings back a “tooling” account that initially I was hoping to replace with CodeCatalyst.
The second thing that I missed was the possibility to automatically map existing roles to groups in Amazon CodeCatalyst – today this is a – one time – manually effort that needs to be conducted. If you need to do that only once for your organization that isn’t a problem, but if you decide to have multiple spaces for your organization and need to perform this multiple times, then this can be very time consuming.
Summary
I believe that this change will have a very big impact on the adoption for CodeCatalyst as administrators and security teams in organizations will make it easier to allow the usage now that there is a central place to manage identities. And with the possibility to actually try out CodeCatalyst in an enterprise world, I hope that we will also see more adoption of CodeCatalyst and with that also a more streamlined and better communicated roadmap of CodeCatalyst. If you would like to learn more on how to activate this, please refer to the documentation.
In this post I’m writing about how getting to re:Invent 2023 was and about how the last year has made a big impact on my personality and my efforts in the AWS community.
Thinking it couldn’t get better last year, 2023 changed me once more and leveled me up
While going to re:Invent 2022 I wrote down my thoughts and went home from re:Invent expecting things could not get better for me. But this year has prooven my expectations wrong, as the last 10 months have been life-changing in a lot of different aspects.
Blogging & Starting a YouTube channel – winning 1.5 hackathons and starting a project
When I reached my home town after last years re:Invent I knew that this community is what makes me happy and gives me back so much that I also would like to give something back. And so, I formed a few ideas in my head and started a few initiatives that I would like to share with you today:
Blogging & Writing
I have been blogging both on dev.to and on my personal blog – writing about topics and things that I do or did on a regular basis, trying to share experiences of day-to-day things and building experiences.
The most important thing for me he is to always be transarent and authentic – speaking out on things that I liked and challenges that I’ve faced.
For you reading this – thank you, for being part of my journey, for giving me feedback and for being you! Please let me know if you have any feedback on what I should change or do better.
Kicking off a YouTube channel
The last three years of my career have been focused on building secure and reliable CI/CD pipelines! But hey, there is so much more to learn. As part of my travel back to my home town after re:Invent, I had the thought to share my experiences and the experiences of other builders on a “podcast-like” YouTube channel which is today known as @cicdonaws.
Learning on how to produce a YouTube video to how to set up a good lightning and sound, preparing a script and creating better thumbnails – a lot of learnings, experiences and hours went into the videos that I have been producing.
This only kicked off because of all of the great builders that joined me on the show and shared their projects, learnings and experiences with me. Thank you all, every single one, for your time, dedication and experiences. Keep up sharing your knowledge, as this is how we can all grow!
Winning 1.5 hackathons and making a small project something big
Back in march Lily announced a, Community Builders-internal hackathon where it was possible to win some prices – and I thought I had a great idea on what to build that actually fit into the “rules” of the hackathon. And this is how the project started that has since then used up countless of hours of my after-work evenings and weekends. Millions of Slack messages have gone from Germany to US, from US and Germany to Africa and back – we kicked of a real project with our Speakers Directory that is finally ready for prime time! After becoming second in the first Hackathon, we also submitted the same project with a few tweaks for a 2nd hackathon powered by Hashnode and Amplify and this also brought us “honourable mentions” in the list of top 10.
We’re ready now to get more of you to add your talks and events! Please sign up and give us feedback, we are very proud of what we have produced so far and eager to get all of you on board!
Thanks to Danielle, Julian, Matt, Raphal, Baimam for actively working with me on this fun project – and thanks to everyone else that has supported us through 2023 to make this a bigger thing!
Organizing a community day – the EU (more technical) edition of re:Invent
Another piece last year has been organizing the AWS Community Day DACH that happened in Munich this year – it was fascinating to be part of the team that brought more than 500 AWS nerds together, including over 25 AWS Heroes and 30 Community Builers!
Thank you all for attending – and thanks for the sponsors for making this possible!
We’re up for doing this again next year – further details to be announced soon! Looking forward to see you there!
Hey, hiking before re:Invent is what you need!
And then, there is this “secret thing”: Ever since I went to re:Invent I used the sunday before re:Invent to go out “hiking” somewhere close to Las Vegas. The first two visits together with colleagues were three persons – a pretty cool small group.
Last year, they did not go with me to re:Invent, so I asked Community Builders to join me – at the end, we were nine persons from I think 8 different countries hiking together from 10 am till 5 pm – we had a lot of fun, so I decided: Let’s do this again!
Somehow, this became a bigger thing – for this years hike, we have over 50 Community Builders, Heros and UG Leaders signed up. The group became so big that the AWS Community team is helping us this year by sponsoring the transportation.
Thank you all, that you are supporting this thing – and thanks to everyone participating, I am sure this will be an amazing experience for everyone!
How and Why what I do for the community changed
Since June I can name myself an AWS Devtools Hero – which honors me a lot but also drove a bit on how and what I do for the AWS community. My work has become a little bit less “public” – a lot of my hours that I have “free” went into talking to AWS Service Teams, into mentoring other builders and helping them grow. This has lead to less new videos on YouTube and less blog posts – but instead, I’ve helped others to grow and to invest into the community – supporting the “re-start” of the AWS UG Frankfurt, the re-start of the User Groups Mainz, Karlsruhe and Heilbronn and overall making other builders “grow”. This makes me happy.Thank you all, that you are part of my journey.
Making new friends
This year has also shown to me how important relationships are and how much you can make friends by supporting each other. Friends, I’ve never met in person (Matt), friends I’ve finally met in September (Ran) and others like Markus, Thorsten, Philipp and Raphael that are part of most of my days and that I write 100s of Slack messages per week.
This is what community means! Making friends, learning, growing. Thank you, for being you!
What’s next for me?
Changing my role at work
I’ve recently moved to the global Platform Architecture team here at FICO and that gives me a lot of new possibility and learning opportunities.
The platform that FICO is building is an important one and helps organizations around the globe to take better decisions – I’m proud to be part of the team and looking forward to make this bigger than it is today.
Community Day DACH 2024
We’re already planning next year. Stay tuned for some announcements…soon!
Supporting and mentoring builders around the globe
I’m up for helping other builders grow – if you’re intersted to collaborate with me or to learn from me, reach out to me and we can talk!
If you have any CI/CD or CodeCatalyst specific topics that you would like to talk about on my channel – reach out to me!
If you need feedback, help or advice on anything (e.g. a blog post or anything else) – let me know and reach out!
Looking forward to an amazing re:Invent 2023 with a lot of friends in Las Vegas!
Have a great week – don’t be a stranger and say HI if you see me around!
The new service announced by Amazon in Las Vegas at re:Invent 2022 which is an integrated DevOps service to empower development teams to develop and deliver software faster finally reaches the “general availability” status. As I have previously outlined, this achievement is very important for Amazon and the CodeCatalyst team. Congratulations to the team for reaching this goal, which I can imagine is not an easy step for this product. The tool touches a lot of very sensitive parts of a software project and I can imagine the security standards being really high.
A hugh achievement – thank you everyone in the team for investing into CodeCatalyst and for listening as closely to the customer feedback as you are!
What changes did get implemented for GA?
As part of the GA release we see a lot of minor improvements in the User Interface and color changes. In the last weeks, we have seen a few “bigger” changes – like the possibility to use Dev Environments for Github based projects. We also got “graviton based” execution environments for CI/CD workflows which, according to AWS, should reduce our costs.
It is still hard to track down all of the changes in CodeCatalyst, as there is – to my knowledge – no public or semi-public roadmap. This is one of the things that I’d love to see, as for an integrated service that is at the core of the Developer Experience for teams, any minor change can either improve or destroy the “usage experience”. If you as a team invest into adoption a new tool like CodeCatalyst, they will need to know how changes in workflow, features or user interface can influence their day-to-day activities. Let’s see, maybe the team can share “something” like a “changelog” with us (or even an RFC process like Amplify or AppSync)?
Reached “GA” – so who can start using it now?
As of today CodeCatalyst is only availble in US regions and this means that it can be adopted mainly by US enterprise customers. CodeCatalyst already gives you the possibility to set up different Spaces for your account and within a space you can manage multiple projects. So in theory, CodeCatalyst is “ready to be used” by everyone.
Practically speaking, it is easier to adapt the service for new projects than for existing projects , as there is no real “import” functionality. Yes, you can integrate existing Github projects, but that only integrates the source code. Unfortunately that does not make all of the “cool” things available right from the start of integrating the source: existing workflows (CI/CD pipelines) are lost and need to be re-build, issues/tickets are not imported into CodeCatalyst (tho they can be made available through the JIRA integration).
I have been regularly using CodeCatalyst (both for imported and “new” projects) – and I really think that the tool already works very well.
The “killer feature” that I see for new projects are the “blue prints” which essentially get you started within minutes, e.g. to deploy a SPA application, or to have a “true” CI/CD pipeline for a full stack application following the DPRA.
Right now I would recommend using CodeCatalyst for any new project that you start to start building out your workflows and best practices.
So what do I still need to recommend CodeCatalyst for existing projects?
There are a few things that I have already been writing about:
“Import” of existing CI/CD workflows (e.g. Github actions, CDK Pipelines or CodePipelines)
Fully import projects
existing issues from Github or JIRA
Git-based projects including the history
Tighter security settings and permissions
Fine granular roles to allow or forbid access to specific parts of a project
Options to allow or forbid execution of workflows (or to deployments)
Additional workflow options
Manual approvals are very high on my wish list
Integration of other AWS services natively
A question for the readers: What do YOU think that you need to adopt CodeCatalyst?
A big question for the CodeCatalyst team – HOW MANY AWS TEAMS ARE USING CODECATALYST FOR PRODUCTION DEPLOYMENTS TODAY?
Where do I see the potential for CodeCatalyst?
CodeCatalyst is a big bet by AWS. There is a big potential that can really improve the life of development teams and these are the main things that I believe that can out-grow other existing solutions:
Integration of AWS Services / deployments metrics
the true integration with AWS APIs
Integration into “post-deployment” verifications (e.g. auto roll-back after failed CloudWatch metrics)
“At-hand” developer support to improve efficiency
with CodeWhisperer (who recently reached GA) AWS already aims to support developers during the development phase, but with CodeCatalyst AWS can take this to the next level:
AI support during Pull Request Reviews (or automated approvals for PRs – e.g. by including CodeGuru, etc., automated merges, etc.)
AI support during workflow executions (when to approve, when to deloy, when to promote, etc.)
With improvement proposals for workflows if the “AI model” recognizes patterns (in issue workflows or CI/CD workflows)
With automated improvements for existing projects based on blue prints
Best practices change – and so blue prints change – and if the CodeCatalyst team can automatically apply them for existing projects, customers will benefit from it
And last but not least:
I trully believe that every software project should start with a CI/CD pipeline – and with the Blue Prints including the CI/CD workflow that follows DPRA and other AWS best practices, we can trully make this possible: Empower developers to deliver their software projects in minutes right after starting their project.
Do you see the potential in CodeCatalyst? If you do not see any potential in the tool – why not?
At re:Invent 2022, as usualy, different new AWS Services or functionalities have been announced in Preview. Now, at the beginning of April 2023, a few of them have already reached the “General Availability” (GA) status – Application Composer (in early march), Latice (in late march). My favourite new service, Amazon CodeCatalyst, has not yet reached this goal – but I have a feeling that now is the right time to think about what and when we can expect this status.
You wonder what CodeCatalyst is? Watch this video on my YouTube channel or read my two initial posts about it.
Why is reaching the “GA” milestone so important?
Before starting with my assumptions on what we can expect for GA, lets clarify why reaching this milestone is so important. Being “in Preview” can mean a lot of different things. In a lot of organizations this usually translate to “limited availability”, a service not being available in all regions or not being reliable or scalable. For other organizations, it means that specific aspects of the product can be immature or not reliable. It can also mean that bigger API change are yet to be implemented or missing security guardrails.
In general, this can be seen as a “beta” offering which is not appropiate to use for productive workloads.
Because of these reasons and maybe others, a lot of organizations (especially US based) do not allow using or adapting services that are in “Preview”.
For all of my experiences, tests, videos and projects I was so far able to only be on the free tier. And I assume that this will also be the truth for most of my readers: You can get a long way using the Free Tier that Amazon CodeCatalyst offers today.
So thats another big reason for AWS to push this service out of “Preview”: It gives organizations that are forbidden to use the service in “Preview” the possibility to start using and adopting the service – and with that Amazon can start earning money with the service which unil now might be difficult.
And as we know, AWS tries to “work backwards” from client requirements and the early usage of CodeCatalyst will drive further investments into the service.
What to expect for GA of CodeCatalyst?
Simple: Nothing big – most probably only regional rollout.
I do personally not expect any major new features for the service as the team has been constantely releasing new features and functionalities to the service on a regular cadance. There was simply not more time to work on bigger features while preparing the “General Availability” (GA).
What the CodeCatalyst already has delivered until today…
Let’s look at what has been added to CodeCatalyst since its official release in december 2022:
Additional Reporting auto-discovery
Change Tracking – the possibility to see which changes have been deployed to a certain environment
Additional Workflow native actions and improvements, E.g.
a problem with the CDK action to be able to define the “workpath” of a CDK app
Additional native actions
Linked issues to Pull Requests – you are now able to link issues to a pull request
UX improvements
Log files wider accessible in UI – at the beginning you where not able to make the log view larger, now this is possible
This is not a complete list, but the things that I personally noticed and that I liked to see.
So…when is “the date”?
Hard to guess, but I would expect “soon”. Ideally right before a month starts, which will make the billing cycle easier 🙂
So I would guess “end of april” which would bring the service right in time for the Berlin Summit (3rd of May).
Next steps for CodeCatalyst
In my last posts I have already been communicating my thoughts and features that I would love to see. But what will AWS implement?
Given reaching the “GA” status opens the way to “enterprise clients” I would expect that one of the first features will be Single-Sign-On functionalities, maybe with an integration to Okta, Ping, Azure Active Directory or other already existing IdPs.
In addition to that I believe that the User Interface needs to get some tweaks to streamline the navigation and workflow – that’s something that I personally experience every day: not knowing when and where to click to get on the rigth place. Also I think that additional service integrations will be added – e.g. StepFunctions or SNS, maybe SQS – see also my post about sending notifications from workflows.
And then there is one last thing which has been getting only limited attention so far: APIs and CLI integrations that can be used – so I would expect a major update there.
I’m really looking forward to see CodeCatalyst reaching GA – I’ve had various conversations with the team in the last months and I know that they have a true vision to make CodeCatalyst successfull as a trully AWS integrated and fully functional DevOps tool.
Are there features you are missing? Please let me know and I will forward them to the team.
As Amazon CodeCatalyst is still in Preview, it has only limited integration possibilities with other AWS services or external tools. Sending notifications from a Workflow execution is something that I believe is critical for a CI/CD system – and as I focus on CI/CD at the moment I’ll focus on the notifications on Workflows in this article.
What kind of notifications do I need or expect?
As a user of a CI/CD and Workflow tool there are different levels of notifications that I would like to receive:
Start / End and Status of Workflow execution
State / Stage transitions (for longer running workflows)
Approvals (if required)
In addition to that, based on the context of the notification I would like to get context-specific information:
a) For the “Start” event I would like to know who or which trigger started the workflow, which branch and version it is running on, which project and workflow has been triggered. If possible getting the expected execution time / finish time would be good b) For the “End” event I would like to know how long the execution took and if it was successful or not. I would also like to know if artifacts have been created or if deployments have been done. If the “End” is because of a failure, I would love to know the failure reason (e.g. tests failed, deployment failed, …) c) For the state transitions I’d love to know the “time since started” and “expected completion time”. I would also like to, obviously, know the state that has been completed and the one that will now be started. d) For approvals I’d love to be able to get the information about the approval ask and all required information (commit Id, branch) to do the approval
What does CodeCatalyst Support today?
Right now CodeCatalyst allows to set up notifications to Slack. Please see details on how to set this up here. This notifications are also minimal right now:
In Slack this looks like this:
How can I enhance the notification possibilities?
Luckily one of the “core actions” is the possibility to trigger a Lambda function and this is what we are going to use here to be able to trigger advanced notifications using Amazon SNS. In our example we are going to use this to send an eMail to a specific address, but you can also use any other destinations supported by SNS like SMS or AWS ChatBot.
Setting up pre-requisites
Unfortunately we will need to set up an SNS topic and a Lambda function in a dedicated AWS account in order to use these advanced notifications. This means that we are “breaking” the concept of CodeCatalyst not requiring access to the AWS Console, but this is the only way that I found so far to be able to send additional notifications.
Ideally we would be setting up the SNS topic and the lambda function using CDK, but that increases the complexity of the workflow and of the setup and because of that I’m not including that in this blog post.
Setting up the SNS topic
Please create a SNS topic following the AWS documentation through the console. We assume the topic to be in “eu-central-1” and the name to be “codecatalyst-workflow-topic“.
You can follow this blog post to manually set up the lambda function through the AWS console, please ensure to give the Lambda functions permissions to use the SNS topic. The required code using Python will look like this:
Obviously the same can be achieved using Typscript, Go or any other supported function. Please adjust the topic_arn to match the topic that you just created. After creation this Lambda function will now have an ARN which should look similar to this: arn:aws:lambda:eu-central-1:<accountId>:function:send-sns-notification-python
We will need this ARN when setting up the notification in our Workflow.
Integration into the workflow
Integrating this Lambda function into a workflow is easy:
As you can see, we are integrating an “aws/lambda-invoke@v1” action which then points to the lambda function that we just created.
In the “RequestPayload” we are passing a few information to the Lambda function which will then be passed to the SNS topic as part of the message. This is how the message will look when received as eMail:
Missing information and next steps for enhanced notifications
As you can see we are able to send notifications from CodeCatalyst to multiple targets, including eMail with this option.
What we are missing is – and I am not sure if thats possible or not – is all of the “metadata” of the workflow execution like:
Workflow-Name
State-Name
Project Name and additional information
In the documentation I was not able to find out the available environment variables about these information…. If you do have any ideas on how to access this metadata, please let me know!
A few weeks ago, on december 1st 2022, Werner Vogels announced Amazon CodeCatalyst. I’ve previously shared my initial thoughts and findings in a blog post. In this post, I’m going to share a few more findings and insights into using Amazon CodeCatalyst and will also see if any of my wishes from the wishlist for CI/CD on AWS have been resolved with CodeCatalyst.
What I have been playing around with…
CodeCatalyst login page
One of my personal projects that I am working on together with a few friends is pegasus-galaxy.net and the CI/CD pipeline that I had built with CDK Pipelines (that I also presented at re:Invent 2022) was the first one to try to move over. In context, we’re talking about a Flutter application for Web running behind CloudFront, deployed using CDK.
I decided to try CodeCatalyst out and go “all in” – and that means moving the code from Bitbucket into CodeCatalyst as well as setting up the other users in CodeCatalyst and moving the workflows (=CI/CD pipelines) over to CodeCatalyst.
CodeCatalyst Menu
In this article I am going to go through each of the sections in CodeCatalyst and will share my experiences, thoughts and findings.
Where I have ideas on how to improve the day-to-day work with the tool, will try to share that.
Before going into details, lets start with the most important thing:
Amazon CodeCatalyst works very well and reliable and the current version of the service is a great foundation for moving all of your CI/CD and development practices to AWS.
The CodeCatalyst team has been very supportive on re:Post, so if you have a question, feel free to ask it there!
CodeCatalyst Overview – Spaces and Projects
Spaces are the “Top-Level” option to organize your CodeCatalyst account. You will need to associate an AWS Account for billing used AWS resources. Each AWS (billing) account can be associated only with one CodeCatalyst Space.
CodeCatalyst Spaces Billing page
While this seems like a limitation as you will need to create a different billing account for a 2nd space, I can right now not see an impact for my day to day work. For anything that I run on the same AWS account, I would assume that using a project within the same space should be good.
You can manage Projects, Members and AWS Account connections on the space page. In the “extensions”, CodeCatalyst currently allows a connection only to the JIRA Cloud. I would expect that additional 3rd party extensions will be supported in the GA version of CodeCatalyst.
Projects Overview and options
A project is a “unit of work” in your product or software that you are building. Within projects, you can manage issues, manage your code repositories, execute workflows (CI/CD) and review report results.
Projects are associated to a Space – and you can create as much projects in a Space as you want. You can add team members to a project, that are not able to access all projects in the space. Unfortunately I have not yet found an option to “hide” projects from Team members that are added on the Space itself.
Managing issues / tickets
CodeCatalyst currently provides two options to manage your issues or tasks: 1) Link to JIRA Cloud Project 2) Internal issue management
If you use the option to link to a JIRA cloud project, the “issues” link is replaced by a link to your JIRA Cloud project.
Internal issue management
The internal issue management system currently offers everything that is required for a simple Kanban workflow. You can create issues, add them to a backlog or a Kanban board, assign them to project members and track their current status. I personally think that the current functionalities are good enough for small teams and simple projects – I’m actually already working with it in a small project and will add additional feedback as soon as I gain more experience.
Code
Within the “Source” part of a project, you can manage source repositories or connections to source repositories in Github. I expect that other providers will be added going forward (e.g. Gitlab, CodeCommit, Bitbucket, …). You can also manage pull requests and approvals – I was only able to test this using internal source repositories, not using a linked repository.
The last option – the Dev Environments – is the most exciting functionality – it gives you the possibility to host development environments (similar to Gitpod) on AWS using Cloud9 but also, and this is really cool, using Visual Studio or JetBrains IDEs. When using that option, the IDE on your local PC is only the “presentation layer”, the source code is stored and run on an AWS instance and the IDE uses remote connectivity to talk to the Dev Environment in the background.
CI/CD
CodeCatalyst currently uses the same approach as Github Actions to manage your workflows or CI/CD pipelines – you are able to manage your Workflows using YAML files. The syntax is simple and understandable. There is a minimal set of Actions available as part of the preview. You are also able to use existing Github Actions as part of your workflow.
Workflow overview in CodeCatalyst
The workflow functionality is very powerful. In my tests I have not yet been able to test all parts of the capabilities. Workflows can be defined for certain directories, for certain triggers or branches. Test reports will be exposed in the “reports” functionality.
CodeCatalyst offers a graphical overview for workflows and alows to edit them in the UI, too. This functionaly works pretty well and helps to quickly get you started building your first workflow in CodeCatalyst.
I’ll need to test the workflows more to be able to give additional insights into how good or bad they are currently running. My simple pipeline that builds a Flutter Application, deploys my Infrastructure as Code using CDK and then publishes the new version of the Flutter app runs without problems.
One of my main concerns so far is the execution time, however the team has been working on a possibility to use Lambda as an execution environment. This option however does not yet support the execution of Github actions and also has some other limitations.
The other features that are part of the “CI/CD” – Environments, Compute and Secrets – I did not have time to play around with this. If you have any experiences with it, please add your thoughts in a comment to this article!
Reports
The reports today only suport test reports. I have not used the functionality enough to assess this, but I am sure that the CodeCatalyst team is going to add additional reporting options going forward.
Things I like most about CodeCatalyst (Preview) after 6 weeks of usage
Just a short list of things that I already like: – Integration of Github Actions as workflow actions – Managing workflows using UI & code
Things I miss in CodeCatalyst (Preview) after 6 weeks of usage
– macOS builds (e.g. for Flutter iOS apps) are still not possible – granular permissions for workflow and Pull Request triggers – and….
Let’s talk about Open Source Projects
Right now there is no option to share a project or a repository that is hosted within CodeCatalyst as an Open Source project. This is really a limitation if you want to use CodeCatalyst for Open Source project – or if I would like to share a CodeCatalyst repository with example workflows. I hope this functionality will be added soon.
Wrap up and next steps for me with CodeCatalyst
I need to admit – writing this post took longer than expected 🙂 I wanted to publish it before christmas and now it seems to be a bit “late” already as I am sure that a lot of you have made your own experiences with CodeCatalyst today – please SHARE your findings with me – links of Blogs that you have written or other content you have created, I am eager to consume it!
My next steps with CodeCatalyst
I am working on migrating my project pegasus-galaxy.net completely to CodeCatalyst and collaborate with my team on it there. With that, I will be able to proof CodeCatalyst in a “real world” application that it is “multi-platform” application – using Flutter for Web, Android and iOS – and a Serverless AWS based backend. If you’re interested to join this project, please do not hesitate to reach out – skills that we need right now: AppSync, DynamoDB and development/software engineering (Flutter, Typescript, Java, or Node?)
As we are now in the post:Invent phase of 2022 and over 10 days have passed since re:Invent 2022 in Las Vegas was concluded, it’s time for a lot of re:Cap Blog posts and events. I’ve read so many of those “major announcments” articles that I’ve decided to write a different type of re:Cap for myself this year: Sharing a few stories from my 10+ days in Las Vegas, as they are as equally important as the technical announcements made by AWS.
Indeed, it was a great conference with a lot of learnings and a lot of very interesting sessions. I focused on Chalk Talks, Builder’s sessions and events (like Gameday) as these are not recorded.
Making new friends before re:Invent kicks off
My flight this year got moved from Saturday to Friday, so I had one more day to get over Jet-Lag. On Friday, I spend a good time shopping and besides that met with Oliver vor dinner. On saturday morning (early morning!), I looked at the AWS Community Builders Slack and found out that Traian set up a “spontanous breakfast” for the Jet-lagged folks – and I ended up sitting over two hours with different parties, having fun, chatting and getting to know people. It was exciting to meet Rafael, who had been our Solutions Architect for a while, for the first time in person – without planning it 🙂 It was also great to meet Heitor in person – the person that owns the Lambda Power Tools at AWS. His talk is now on Youtube and I would encourage you to listen to it if you are interested in Open Source.
The rest of the saturday I spend with Markus, who shared so much Las Vegas knowledge with me that I think my brain is still burning – and I would not dare to claim remembering more than half of what we discussed – but it was a great saturday which ended with meeting Philipp for dinner at “The Cheasecake factory”.
I kicked off the sunday with a lot of excitement about my very first talk at re:Invent – final technical check in the “Speaker Ready Room” for my slide deck! That needed to be early morning, because afterwards I had planned to go out hiking with fellow AWS Community Builders.
Hiking across time zones and cultures
Definately one of my highlights this year: The ever first AWS Community Builders pre:Invent Hiking Trip!
Thanks to everyone that joined – Oliver, Richard, Jenn, Ganesh, Traian, Pubudu, Niklas. It was great to see how we supported each other, had great conversations and all managed to get across different challenges we had to fight!
Thank you Oliver (and kreuzwerker) for the amazing video. Traian, you’re my hero. Congratulations on finishing off the hike with us with. Thats an achievement noone can take away.
It was fascinating to meet you all for the first time and notice that we get a long well, without ever meeting before. That’s the power of the AWS community!
We got back at 6pm, after a over 5 hours hike, just in time to get our AWS re:Invent badges and to meet other AWS Community Builders from around the globe for a great dinner. Lilly & Jason – thanks for joining us, that really made me happy!
Welcome to re:Invent
Kicking off re:Invent with a GameDay with a great team an Jeff Barr
I decided to kick off my re:Invent on Monday with a GameDay – which is a fascinating opportunity for gamified learning. On sunday, during our hike, I had aligned with Niklas to form a team together – and the other two team members, JaeJun and Martin, we met in the morning. We had great fun, ended up 4th even tho Jeff Barr distracted us for some time as we won him on our table with a quizz. It was great meeting him in person – and I can tell you: He is a human as we are, even if his Newsblog is legendary 🙂
Meeting people from the AWS Community
I had so much great hours in Las Vegas – thanks for the time spend together, everyone that I’ve met – Stefanie, Oliver, Manuel, Mike, Stefan, Philipp, Markus, Thorsten – and others – from the german community.
Finally met Danielle and Matt in person. Another of my highlights.
The Community Builders Mixer and the User Group Leader Mixer where both great events to get to know each other better and network with great people from everywhere in the world. I met so many people that I had been interacting with in written (Slack, Twitter, LinkedIn) – it was a blast for myself.
Famous Jenga-game with AWS Heros – so much fun!
Famous Jenga-game with AWS Heros – so much fun!
Speaking at re:Invent 2022 – my DevChat
As I’ve already shared before this year I had the opportunity to speak at re:Invent – COM307 – Using CDK pipelines (in Java) to build a multi-platform Flutter application
Thanks for everyone that made this possible: Ernesto, Shantavia, Lilly, Jason, Maria! It was my biggest honor to share my experiences and my open source initiative. Looking forward to keep sharing knowledge!
Announcing Amazon CodeCatalyst
With the announcement of Amazon CodeCatalyst the conference brought for me a new service that I am eager to use and try out as I am very much interested in CI/CD on AWS. This was for me definately the most exciting anouncement of re:Invent 2022 and I had a lot of interest to talk to the service team, product managers and others after the service had been announced. I’m looking forward to share more about that as I get to play around with it more.
Flying out
On saturday my trip to re:Invent was over and it ended as my re:Invent trip began: meeting AWS interested persons at the airport (Thanks Maria for the introduction!) and with great conversations with Oliver on the way back to Frankfurt.
Thanks to everyone that I met and talked to at re:Invent 2022 – you really made this conference be a different one for me than it was before.
I’m looking forward to hopefully meet all of you again in 2023!
