Course Builder: Commerce Port from Skill Stack Gets Stubbed Out
- Invoices face issues with visibility due to incorrect keying and permission settings.
- Elements moved from 'pages' to 'app' directory for easier integration and maintenance.
- TRPC connectivity issues were encountered, leading to manual configurations until stable network conditions were re-established.
- Commerce functionalities neatly categorized under the 'app' directory, aiding in better organization and focus.
- Dynamic loading implemented for error management and better component performance.
- Consideration of using consistant adapter patterns for payments for flexibility and scalability.
- Backend logic efficiently handles complex scenarios like new purchases and coupon applications.
- Proposal to record webhook events in a dedicated table for enhanced traceability and system integrity.
- Future developments include optimizing component packaging and enhancing reusability.
Migrating the commerce module of the Course Builder, several changes and updates have been implemented. First, let's discuss the issue with viewing invoices, which appears to not function as anticipated due to a naming mismatch in the creator's name used as a key. Despite being logged in, there seems to be an unexpected redirection to the login page due to outdated user permissions, and these issues have been flagged for debugging.
Significant restructuring and additions in the codebase include moving components away from the 'pages' directory to simplify integration directly into the 'app' directory, particularly for server-related components. However, TRPC was not functional under certain network conditions, necessitating a temporary workaround until the connection was stable again.
The commerce-related functionalities are broken down within the 'app' folder structure, separating invoices and purchase-related processes to streamline development and debugging. The system now dynamically loads components to manage potential errors effectively, keeping the core functionality isolated within the 'app' directory, specifically under 'commerce' for better organization.
Moving forward, there's a discussion on the Stripe payment adapter's integration or whether to capture charges directly, which leans towards using a consistent adapter pattern for flexibility. The backend handles complex business logic, ensuring that various conditions like coupon applications and new purchases are managed efficiently. Decoupling and testing these processes will be crucial for long-term maintenance and scalability.
Finally, there are ideas on improving system robustness by recording webhook events in a dedicated external events table for traceability and forensic purposes. This, alongside the ongoing adjustments to how components are packaged and reused, will shape the next development phase to optimize and fortify the commerce functionalities within the Course Builder system.
Transcript
[00:01] All right, so this is going to be just an overview of some of the commerce additions that got made here. So let's see, here we go. And so here's the post purchase, Obviously I haven't, this is all just yoinked from total TypeScripts and you can see the blue. And it's only dark mode over there. So this all looks goofy, but you have transfers, et cetera.
[00:30] Let's see, you can hopefully come over here. And you get your invoice. It's kind of broken, that's fine. That's because I don't have like a creator name and it's keyed to that. So you still get your transfer to purchase.
[00:47] I don't know, does this work? Yeah, nice. You should also get the list of invoices I actually haven't tried that URL so we'll see if that works it didn't I am logged in I don't know why it doesn't work, but add that to the list of things to fix. But I thought I'd kind of go through what occurred to get here and kind of where I need to go mostly just to think about it out loud and figure out what to do next. So this is all just in Course Builder and I plopped most of it down into the top level folders and then I tried to bring it over just as pages to make it simpler.
[01:36] So I didn't have to do all of the server component stuff and app writer stuff, but for whatever reason, TRPC did not work when I was in the hotel Wi-Fi for whatever reason. I have no idea. It started working again later when I was in airport Wi-Fi. But what that means is that I had to go through and add this in the app directory. So there is no pages directory and at the end of the day that's fine it's actually a pretty good replacement for most of the TRPC stuff so it is all contained inside of commerce so that's the parenthetical directories inside of the app router are nice because they are only there for organization.
[02:21] This is ignored and this is as if invoices was in the app folder, but we're able to sort them into commerce, content, email list, user, etc. So that makes it pretty simple to come in and just grab all of or think about all of the commerce stuff and you know so you come in here and you have invoices and I did do some breaking out a little bit You'll notice that a lot of stuff is in path to purchase and purchase transfer. So I got that going, but most of this is just kind of directly ported and it's all server. Like this didn't, I don't know. Yeah, like the invoice print button needed to be a client component.
[03:07] You can see all these aren't filled out on this site. So that's why it looks all weird. But I mean, otherwise, custom text, I went ahead and made that because that's using the local storage so that's a client component but otherwise this is a server component and we just load data and format it in it and it works just fine these I don't know what was going on, but they kept trying, these failed. So this makes sure that it loads it dynamically. So that's why you'll see the headers being loaded and not doing anything.
[03:40] I've noticed Vercel do that in some of their stuff. I don't know exactly why it's kind of a chicken bones, voodoo kind of thing. Invoices, we saw this one not working. If ability can view invoice. Interesting, I can probably just fix that actually All right, I'll fix it later Yeah, so that that's what it's doing.
[04:06] I just can't view invoice, so it's redirecting me to login, but I'm already logged in, so that doesn't make any sense. And that's just because I have not updated the abilities. I was wondering where that... Server off station, get ability, create ability, get ability rules. I would assume can manage all is the, gonna allow that, but maybe not.
[04:36] Anyway, I'm not gonna fix that now. So pricing here, that is Actually, well, that's not a component at all, honestly, or it's not a route. It's fine in here, but this is not probably where it goes. I hate to just pass it up, so I'm going to move that. And this will end up going to app.
[05:15] More app source. And probably path to purchase. That should under the hood also update any imports that I had. Cool. So this thanks page, Thanks purchase is our big one.
[05:32] We also have redeem. Also if you redeem a golden ticket or a free thing takes you to redeem but here or Maybe a team thing too But here this is a little more complicated. I just kept get server-side props as the method name here and it takes a session ID which comes in as search param and then getServerSideProps just got converted over so I use the payment provider, uses the course builder adapter and you can just import that around so any place in Skillstack where you had to get SDK for the most part you can bring in a course builder adapter and it works just fine and then that returns probably doesn't need to serialize this anymore. I'm not 100%, I didn't check that, but might not be the case that you need to serialize this because it's not, it's different. This is all run on the server, so on the page, again, I do the headers, and then we come in here and just grab that and pull all of those off of there.
[06:37] It works fine with the serialization, I just don't know if it's necessary or not. You probably, like I would assume, lose, yeah, that becomes any, right? So, which is fine just to like like kind of hack it in but I feel like this is kind of weird and then all that gets pushed into thanks verify and I think at the time I thought that I might need to pull this out and make it because it was using TRPC so the plan was to make this into a client like a used client from the top but as it happens it just works so after I had it was having trouble with TRPC and had to do all of the things I ended up in a place where I didn't need to do that. These are the components. Some of these are...
[07:20] They get colored differently. I don't know that any of these ended up needing to be explicitly used client components. And I did add some. I don't remember exactly all of where they are. But yeah, so this works and purchase transfer works as expected.
[07:38] Purchase transfer status, down to the form, right? Like, so when we get down to the form level of the purchase transfer, this uses a little bit of state and is client-side. So it is useClient. That's pretty deep into it and it just kind of worked up here so that was good. Same with the redeem which has not been tested yet and again the welcome page is pretty much it's very like the purchase thanks page but different enough that it deserves its own route obviously and will even redirect to thanks as needed.
[08:17] So what are the other parts? So the other parts end up actually being, let's see, course builder. There's this Stripe payment adapter, And this is where we kind of built this interface and we can get different things. There are places, so this, I'm gonna find it. So like on the merchant charge page, I didn't add these to the Stripe just because I was, frankly, I was just kind of tired and it needed to be specific.
[08:55] So instead of adding these to the provider adapter, I dropped Stripe in. And that's one of the kind of the nice things actually long-term you want to not do that it would be way better to you know add that to the Stripe adapter and then access that versus accessing it this way where we're kind of diving in and using Stripe, it's better to use the provider interface because then retrieving charges would be not specific to Stripe. And then expansions are also kind of an issue, I think, and something that we probably need to avoid and stripe and load the things in sequence versus expanding them or just using the IDs where appropriate, which is the case a lot of times, including line items down here. So I did that in a couple of places. The other way to do that that you'll notice sometimes is that you in the consumer app inside of whatever is consuming I have access to the database directly I don't need to go through an adapter but I think going through an adapter is ultimately going to be better, right?
[10:06] Like, so this is get merchant session, which just doesn't exist on the adapter. And the reason I do this is absolutely pure laziness. And I use air quotes with laziness, it's just like there's a lot going on and I'm trying to remove variables but need to understand that this has to absolutely 100% be moved to the adapter. It just doesn't make sense otherwise long-term because we need to port this functionality across systems and the more the adapters are used the more portable that it will ultimately be. But you have to come down here and you have to update interfaces and add these methods to various places and it's just a kind of a chore.
[10:51] So I'm deferring the chore. I think technical debt is what they call that. So anyway, you get down to the invoices and then it's just kind of Bada bing, bada boom. So the Stripe adapter, and then a big place that this ends up getting updates is like a lot of the, there's this, Where is it? Create charge.
[11:18] Create merchant charge and purchase. Like this is a huge one. It doesn't actually have any tests right now and that would be wonderful. Like that's something that we should totally consider. It does a lot in here.
[11:30] And this is all kind of on our side where we're taking that and we're just assembling the database objects and linking everything and figuring out was this an existing coupon and if this is an existing coupon then we need to do stuff so all of this is like yeah I made a little testing matrix in Figma. Over here, it's like all this stuff that just needs to be tested. And this is probably a good place to use like end-to-end test, that sort of thing. You know, like it's just a lot, like this is just a lot of logic and it gets into PPP and new purchases and you know, just a bunch. And ultimately you return the purchase that was now just created and then you do stuff with it.
[12:25] But so that that's a super important one and all of that is built not in the database adapter but inside of core, source And we look at ingest. I made Stripe, so we have, that happens in here, right? So we get Stripe checkout session completed that exists inside of ingest, and that is then configured and added to the available functions inside of the consumer app. And it just goes through and loads the merchant account, loads the checkout session, gets all that going, finds or creates the user, and then just kind of steps through that. And finally, we see this monster here that then gives us the ultimate purchase that we're looking for.
[13:15] And the purchase being created is, and then it's recorded in the database. So once this is complete, we're good to go. So at each of these, we don't need to necessarily test the ingest function, but testing the logic inside of create merchants and purchase and then breaking that down such that it makes more sense, both to look at, but then makes it testable from a logic standpoint and not necessarily a full integration test because I think Stripe in particular is gonna be challenging there, but our surface area is pretty small. We only take four Stripe events in general. Like that's the only webhooks we listen to.
[13:51] So I think we can probably like reduce the surface area and use just JSON mock files when we're running integration tests against our own database, which should be pretty robust and give us some good test coverage. Haven't done that yet. That's on the horizon. I'm looking forward to it, actually. Let's see.
[14:11] So that comes in and we're in core. And I just have to remember a little bit where everything happens in here. Lib. So inside of core you get lib and the index of lib is actually kind of the router. This is what's called course builder internal and this is take a request from anywhere return it into a request and it has our CourseBuilderConfig which is our options which has all sorts of good stuff on it.
[14:42] This gets initialized, the providers are parsed, it decides which provider is in play here. This is decided by forward slash API, forward slash course builder, forward slash action. And in this case, we're looking for the webhook action, forward slash action, forward slash provider. So forward slash webhook, forward slash stripe is the stripe provider so the action comes in here the provider is attached as options the internal options those actually know they have a provider type so the provider type here in this case is webhook. Great, so then we come into here and we have case, actually the provider type here is payment and the webhook.
[15:28] So you could have different... Right, So we get webhooks from transcription services or payment providers. And what's interesting to me is that while this does say Stripe, because it's the provider type, if we had PayPal or Lemon Squeezy or whatever, you just look at the ID and then you can use whatever handler is correct for that particular payment type with the webhook and then the provider ID so then you know which provider you're using and then you can decide what it is what you want to do. And then this gets into, and notice I'm not even using these adapters in here, which I thought was interesting. And that's because it pitches it out to ingest.
[16:11] That's all it does, right? Like it comes in here and parses it. So it parses the checkout session event with a schema and the data schema. So then it's actually typed which is pretty nice and you get... So this is the checkout session basically the data.
[16:32] So the checkout session completed event and then it actually puts that in there. So I'm just forwarding that and not doing anything and that's what I would do for the rest of them too. So you're not digging around in here you are pushing that over to our ingest function. One thing to note, it might be interesting to do a little before it gets sent into ingest, might be interesting to record the event or the base object to the database. Just write it to, basically write the stripe event or have an events, like an external events table that you could, we could write to that would would store all of these events in mostly in a forensic way.
[17:19] So if something happened down the line, we'd be covered. Trying to think what else got added here. Let's look at it from this perspective. Oh, we got one thing failing. That's not too bad.
[17:41] Don't know which one it is. Doesn't really matter. Back over here. So there's a lot of changes, but... API...
[17:51] Over here. So there was a lot of changes, but API removed all the edge rendering from this entirely because Vercel says you don't need it anymore so cool. Some TRPC churn, adding components, course builder config that adds the email provider which I have actually deferred for now and we'll be getting back to because this needs to send emails Another thing that we can do is remove the planet scale serverless driver. That's kind of a big to-do. A lot of this has is all these components and all this stuff and path to purchase and purchase transfer to a lesser extent.
[18:51] And then of course, team. So these are all related really, like this is all commerce related stuff and could probably all go in like one thing. So I don't know, it's like, This feels to me kind of like commerce react or maybe just a react package that gets Compiled like where does this stuff go? Like I don't What I don't want to do and what I want to avoid doing is putting these in the UI package because I feel like these can be more portable than or external to the mono repo anyway. And I don't know how to do that yet so I'm still trying to figure that out some minor updates to ProAWS just because I'm changing stuff and then what I change in here just yeah this is distracted work This needs to get merged so I can stop doing that.
[19:51] Yeah, anyway, so that's the overview.