{"id":232908,"date":"2023-12-21T11:24:09","date_gmt":"2023-12-21T09:24:09","guid":{"rendered":"https:\/\/isletgroup.fi\/?p=232908"},"modified":"2024-02-22T16:12:35","modified_gmt":"2024-02-22T14:12:35","slug":"microsoft-fabric-brings-new-possibilities-to-lakehouse-implementations","status":"publish","type":"post","link":"https:\/\/isletgroup.fi\/en\/2023\/12\/21\/microsoft-fabric-brings-new-possibilities-to-lakehouse-implementations\/","title":{"rendered":"Microsoft Fab\u00adric brings new pos\u00adsi\u00adbil\u00adi\u00adties to Lake\u00adhouse implementations"},"content":{"rendered":"<p>[et_\u200bpb_\u200bsection fb_built=\u201c1\u201d _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d da_disable_devices=\u201coff|off|off\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d da_is_popup=\u201coff\u201d da_exit_intent=\u201coff\u201d da_has_close=\u201con\u201d da_alt_close=\u201coff\u201d da_dark_close=\u201coff\u201d da_not_modal=\u201con\u201d da_is_singular=\u201coff\u201d da_with_loader=\u201coff\u201d da_has_shadow=\u201con\u201d][et_pb_row _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d][et_pb_column type=\u201c4_4\u201d _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d][et_pb_text _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d]<\/p>\n<p><strong>In recent years, Data Lake\u00adhouse archi\u00adtec\u00adture has become the pri\u00adma\u00adry data archi\u00adtec\u00adture for cloud-based data plat\u00adforms. The Medal\u00adlion archi\u00adtec\u00adture (bronze, sil\u00adver, gold) has become the de fac\u00adto stan\u00addard when build\u00ading a&nbsp;Lake\u00adhouse. Until now, Microsoft\u00ad\u2019s solu\u00adtion for cloud-based data plat\u00adforms has been the Azure Synapse Ana\u00adlyt\u00adics PaaS solu\u00adtion or Data\u00adbricks on Azure. <\/strong><strong>Novem\u00adber 2023<\/strong><strong> Microsoft released a&nbsp;new SaaS-based ana\u00adlyt\u00adics plat\u00adform called Microsoft Fab\u00adric. If Fab\u00adric is not yet famil\u00adiar to you, check out the <a href=\"https:\/\/learn.microsoft.com\/en-us\/fabric\/get-started\/microsoft-fabric-overview\" target=\"_blank\" rel=\"noopener\">Microsoft Learn overview<\/a>.<\/strong><\/p>\n<p>Below are some of our thoughts on Fab\u00adric relat\u00aded to Data Lake\u00adhouse imple\u00admen\u00adta\u00adtions and why orga\u00adni\u00adza\u00adtions should adopt Fabric.<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d]<\/p>\n<h2><strong>Synapse vs Fabric<\/strong><\/h2>\n<p>The pur\u00adpose of Synapse Ana\u00adlyt\u00adics was to bring the data ser\u00advices avail\u00adable in Azure under one umbrel\u00adla. In prac\u00adtice it did, but under the hood they did not always work togeth\u00ader seam\u00adless\u00adly. Exam\u00adples of this include Spark and SQL work\u00adloads not native\u00adly com\u00admu\u00adni\u00adcat\u00ading with each oth\u00ader, con\u00adnec\u00adtions between dif\u00adfer\u00adent ser\u00advices not work\u00ading out of the box, and need\u00ading to store mul\u00adti\u00adple copies of data to opti\u00admal\u00adly uti\u00adlize it in dif\u00adfer\u00adent work\u00adloads. These are not insur\u00admount\u00adable issues, but they cause extra work that does not add val\u00adue for end users. Of course, many of these things can be man\u00adaged with automa\u00adtion, as the tem\u00adplate-based solu\u00adtions we have devel\u00adoped at Islet do. In both solu\u00adtions, Spark note\u00adbooks which per\u00adform the actu\u00adal data han\u00addling, are at the heart of the Lake\u00adhouse archi\u00adtec\u00adture. For this pur\u00adpose, we have devel\u00adoped our own Spark libraries, which make imple\u00admen\u00adta\u00adtion faster and more qual\u00adi\u00adty-con\u00adscious, and they are ful\u00adly com\u00adpat\u00adi\u00adble with Fab\u00adric\u2019s notebooks.<\/p>\n<h2><strong>How does Fab\u00adric change the picture?&nbsp;<\/strong><\/h2>\n<p>Fab\u00adric does not change the basic prin\u00adci\u00adples of the Data Lake\u00adhouse and Medal\u00adlion archi\u00adtec\u00adture, but it pro\u00advides a&nbsp;com\u00adplete\u00adly new type of plat\u00adform for build\u00ading them. Since it\u2019s a&nbsp;SaaS ser\u00advice, its set\u00adup and main\u00adte\u00adnance require less work than the PaaS-based Synapse.<\/p>\n<p>Fab\u00adric\u2019s com\u00admon inter\u00adface for all work\u00adloads is a&nbsp;good thing, of course, and reduces the num\u00adber of tools need\u00aded and the tran\u00adsi\u00adtion between them. As for the work\u00adloads, Fab\u00adric has a&nbsp;lot to choose from: Data Fac\u00adto\u00adry, Data Engi\u00adneer\u00ading + Lake\u00adhouse, Data Ware\u00adhouse, Data Sci\u00adence, Real-time Ana\u00adlyt\u00adics, Data Acti\u00adva\u00adtor, and Pow\u00ader BI. You can read more about these from the Fab\u00adric intro\u00adduc\u00adtion found in the afore\u00admen\u00adtioned link. Of course, not all work\u00adloads need to be used, but the most suit\u00adable tool is cho\u00adsen for each need. The same things can be imple\u00adment\u00aded with dif\u00adfer\u00adent work\u00adloads, for exam\u00adple, alter\u00adnate imple\u00admen\u00adta\u00adtions in a&nbsp;low-code or code-first manner.<\/p>\n<p><strong>How\u00adev\u00ader, the most impor\u00adtant fea\u00adture is the One Lake stor\u00adage space under the hood and the Apache Delta Lake stor\u00adage for\u00admat used by all Fab\u00adric work\u00adloads<\/strong>. Behind One Lake is Azure Data Lake Stor\u00adage Gen2, which means that One Lake sup\u00adports all the same fea\u00adtures as Data Lake Stor\u00adage. Delta Lake, on the oth\u00ader hand, is an open stor\u00adage for\u00admat that sup\u00adports ACID trans\u00adac\u00adtions and data ver\u00adsion\u00ading, and the same for\u00admat is also used by Data\u00adbricks. Synapse\u2019s Note\u00adbooks can just as well use Delta Lake, and Server\u00adless SQL Pool can also read it, but <strong>in Fab\u00adric all work\u00adloads both read and write Delta Lake native\u00adly<\/strong>. This of course makes it eas\u00adi\u00ader to uti\u00adlize data between dif\u00adfer\u00adent work\u00adloads and also the abil\u00adi\u00adty of peo\u00adple in dif\u00adfer\u00adent roles to uti\u00adlize the data on the plat\u00adform, which is exact\u00adly what a&nbsp;mod\u00adern data plat\u00adform should be.<\/p>\n<p>With the uni\u00adfied Delta Lake for\u00admat, the need to copy the same data in dif\u00adfer\u00adent for\u00admats for dif\u00adfer\u00adent tools or use cas\u00ades is sig\u00adnif\u00adi\u00adcant\u00adly reduced. In addi\u00adtion to this, Fab\u00adric has com\u00adplete\u00adly new fea\u00adtures like short\u00adcut and data\u00adbase mir\u00adror\u00ading, which allow exist\u00ading data, for exam\u00adple from AWS\u2019s S3, Azure\u2019s Stor\u00adage, or Azure\u2019s SQL and Snowflake data\u00adbas\u00ades, to be linked to One Lake with\u00adout nec\u00ades\u00adsar\u00adi\u00adly need\u00ading to be sep\u00ada\u00adrate\u00adly trans\u00adferred to One Lake. Each case should of course be stud\u00adied in more detail and the most suit\u00adable solu\u00adtion sought for the spe\u00adcif\u00adic&nbsp;need.<\/p>\n<p>Among the new fea\u00adtures, worth men\u00adtion\u00ading sep\u00ada\u00adrate\u00adly is the Pow\u00ader BI\u2019s Direct Lake con\u00adnec\u00adtor, which can read data from One Lake in real time and very effi\u00adcient\u00adly, essen\u00adtial\u00adly com\u00adbin\u00ading the best aspects of Direct Query and Import Mode type con\u00adnec\u00adtions: up-to-date infor\u00adma\u00adtion mod\u00adel and efficiency.<\/p>\n<p>In addi\u00adtion to the above, Fab\u00adric has numer\u00adous oth\u00ader new fea\u00adtures and the prod\u00aduct is con\u00adtin\u00adu\u00adous\u00adly devel\u00adop\u00ading. It\u2019s impor\u00adtant to note that although Microsoft released a&nbsp;pro\u00adduc\u00adtion-ready (GA) ver\u00adsion of Fab\u00adric in Novem\u00adber 2023, there are still defi\u00adcien\u00adcies in its fea\u00adtures. How\u00adev\u00ader, these are being patched at a&nbsp;rapid pace and new fea\u00adtures are being announced weekly.<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d hover_enabled=\u201c0\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d sticky_enabled=\u201c0\u201d]<\/p>\n<h2><strong>When is a&nbsp;good time to start using Fabric?&nbsp;<\/strong><\/h2>\n<p>Orga\u00adni\u00adza\u00adtions that are just start\u00ading to tran\u00adsi\u00adtion to a&nbsp;cloud-based data plat\u00adform should def\u00adi\u00adnite\u00adly con\u00adsid\u00ader Fab\u00adric as a&nbsp;pri\u00adma\u00adry option. On the oth\u00ader hand, those orga\u00adni\u00adza\u00adtions that have already built their data plat\u00adforms on Synapse or Data\u00adbricks are in no hur\u00adry to trans\u00adfer already com\u00adplet\u00aded parts to Fab\u00adric, as Synapse will con\u00adtin\u00adue to be ful\u00adly sup\u00adport\u00aded. How\u00adev\u00ader, for these orga\u00adni\u00adza\u00adtions, it may be an inter\u00adest\u00ading option to imple\u00adment Fab\u00adric for a&nbsp;cer\u00adtain area of use and thus gain expe\u00adri\u00adence with the new platform.<\/p>\n<p>There are indi\u00adca\u00adtions that Microsoft will pro\u00advide tools for migra\u00adtions at some point. If an orga\u00adni\u00adza\u00adtion\u2019s cur\u00adrent Synapse-based solu\u00adtion is Lake\u00adhouse using Spark note\u00adbooks, as Islet\u2019s imple\u00admen\u00adta\u00adtion mod\u00adel is, the migra\u00adtion to Fab\u00adric will be a&nbsp;fair\u00adly light oper\u00ada\u00adtion, regard\u00adless of whether it hap\u00adpens now or in a&nbsp;few years.<\/p>\n<h2><strong>In sum\u00adma\u00adry, what ben\u00ade\u00adfits does Fab\u00adric bring to an organization?<\/strong><\/h2>\n<p><strong>Under the same ser\u00advice, you can now find every\u00adthing relat\u00aded to data and ana\u00adlyt\u00adics needs<\/strong> from data inte\u00adgra\u00adtion to its mod\u00adi\u00adfi\u00adca\u00adtion, stor\u00adage, and report\u00ading, as well as machine learn\u00ading and AI&nbsp;tools.<\/p>\n<p>Since all Fab\u00adric tools rec\u00adog\u00adnize the cen\u00adtral\u00adized One Lake and use the same data stor\u00adage for\u00admat, it\u2019s easy for peo\u00adple in dif\u00adfer\u00adent roles to uti\u00adlize the infor\u00adma\u00adtion stored on the plat\u00adform. Time and mon\u00adey are saved when an indi\u00advid\u00adual does not have to fig\u00adure out how to read the desired data.<\/p>\n<p><strong>Like\u00adwise, work is made more effi\u00adcient by Copi\u00adlot<\/strong>. It is inte\u00adgrat\u00aded as part of all Fab\u00adric work\u00adloads and has the same vis\u00adi\u00adbil\u00adi\u00adty to the data on the plat\u00adform as the devel\u00adop\u00aders, so devel\u00adop\u00aders can ask Copi\u00adlot to write code, cal\u00adcu\u00adlate for\u00admu\u00adlas, or ana\u00adlyze data, for example.<\/p>\n<p>Fab\u00adric\u2019s costs are based on capac\u00adi\u00adty units, which all work\u00adloads con\u00adsume. As the amount of data grows and usage needs expand, more capac\u00adi\u00adty is pur\u00adchased or vice ver\u00adsa. Pow\u00ader BI licens\u00ades, how\u00adev\u00ader, are still pur\u00adchased sep\u00ada\u00adrate\u00adly unless using the F64 capac\u00adi\u00adty, ie. the for\u00admer Pow\u00ader BI Premium.<\/p>\n<h2><strong>Islet and Fabric&nbsp;<\/strong><\/h2>\n<p>At Islet, we have been imple\u00adment\u00ading Data Lake\u00adhouse archi\u00adtec\u00adtures for a&nbsp;long time instead of tra\u00addi\u00adtion\u00adal data ware\u00adhous\u00ades, as in the case of <a href=\"https:\/\/isletgroup.fi\/en\/2023\/11\/08\/wihuri-transforms-data-management-and-analytics-with-islet\/\"><strong>Wihuri<\/strong><\/a>. We have devel\u00adoped gener\u00adic, repeat\u00adable mod\u00adels and libraries for the effi\u00adcient imple\u00admen\u00adta\u00adtion of the Medal\u00adlion archi\u00adtec\u00adture and use Delta Lake as the data stor\u00adage for\u00admat. Con\u00adsid\u00ader\u00ading these, the tran\u00adsi\u00adtion to Fab\u00adric does\u00adn\u2019t great\u00adly change our way of imple\u00adment\u00ading Lake\u00adhouse, but it brings many new pos\u00adsi\u00adbil\u00adi\u00adties and fea\u00adtures for build\u00ading the data plat\u00adform and uti\u00adliz\u00ading&nbsp;data.<\/p>\n<p>[\/et_pb_text][et_pb_text _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d hover_enabled=\u201c0\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d sticky_enabled=\u201c0\u201d]<\/p>\n<p>-\u2009\u2014\u2009-&nbsp;\u2014&nbsp;-<\/p>\n<p><em>The blog\u2019s author<span class=\"ui-provider a b c d e f g h i j k l m n o p q r s t u v w x y z ab ac ae af ag ah ai aj ak\" dir=\"ltr\"> Mika Kuiv\u00ada\u00adnen is data archi\u00adtect at Islet with over 15&nbsp;years of expe\u00adri\u00adence about data\u00adbas\u00ades, data &amp;&nbsp;ana\u00adlyt\u00adics and consulting.&nbsp;<\/span><\/em><\/p>\n<p>More info:<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=\u201c1_4,1_4,1_2\u201d _builder_version=\u201c4.16\u201d background_size=\u201cinitial\u201d background_position=\u201ctop_left\u201d background_repeat=\u201crepeat\u201d global_colors_info=\u201d{}\u201d][et_pb_column type=\u201c1_4\u201d _builder_version=\u201c4.16\u201d custom_\u200bpadding=\u201d|||\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d custom_padding__hover=\u201d|||\u201d][\/et_pb_column][et_pb_column type=\u201c1_4\u201d _builder_version=\u201c4.16\u201d custom_\u200bpadding=\u201d|||\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d custom_padding__hover=\u201d|||\u201d][et_pb_image src=\u201chttps:\/\/isletgroup.fi\/wp-content\/uploads\/2021\/02\/Janne_Anttila_Beanie_Round.png\u201d title_text=\u201cJanne_Anttila_Beanie_Round\u201d url_new_window=\u201con\u201d align=\u201ccenter\u201d align_tablet=\u201ccenter\u201d align_\u200bphone=\u201d\u201d align_last_edited=\u201con|desktop\u201d _builder_version=\u201c4.16\u201d width=\u201c85%\u201d global_colors_info=\u201d{}\u201d][\/et_pb_image][et_pb_text _builder_version=\u201c4.17.4\u201d custom_margin=\u201d|-10px||-30px|false|false\u201d global_colors_info=\u201d{}\u201d]<\/p>\n<p style=\"text-align: center;\">Janne Antti\u00adla<\/p>\n<p style=\"text-align: center;\">CBO\u2009\u2014\u2009Data and Ana\u00adlyt\u00adics, Isletter<\/p>\n<p style=\"text-align: center;\"><a href=\"mailto:janne.anttila@isletgroup.fi\">janne.\u200banttila@\u200bisletgroup.\u200bfi<\/a><\/p>\n<p style=\"text-align: center;\">+358 45 672&nbsp;8569<\/p>\n<p>[\/et_pb_text][\/et_pb_column][et_pb_column type=\u201c1_2\u201d _builder_version=\u201c4.16\u201d custom_\u200bpadding=\u201d|||\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d custom_padding__hover=\u201d|||\u201d][et_pb_text admin_label=\u201cmore infor\u00adma\u00adtion text (fill it if you want to\u201d _builder_version=\u201c4.16\u201d custom_margin=\u201d||40px|||\u201d global_colors_info=\u201d{}\u201d][\/et_pb_text][et_pb_button button_url=\u201cmailto:janne.anttila@isletgroup.fi\u201d url_new_window=\u201con\u201d button_text=\u201cGet in touch\u201d button_alignment=\u201ccenter\u201d admin_label=\u201cemail\/get in touch but\u00adton\u201d module_class=\u201cCTA_box_bedrock\u201d _builder_version=\u201c4.17.4\u201d custom_margin=\u201c140px||||false|false\u201d global_\u200bcolors_\u200binfo=\u201d{}\u201d button_text_size__hover_enabled=\u201coff\u201d button_one_text_size__hover_enabled=\u201coff\u201d button_two_text_size__hover_enabled=\u201coff\u201d button_text_color__hover_enabled=\u201coff\u201d button_one_text_color__hover_enabled=\u201coff\u201d button_two_text_color__hover_enabled=\u201coff\u201d button_border_width__hover_enabled=\u201coff\u201d button_one_border_width__hover_enabled=\u201coff\u201d button_two_border_width__hover_enabled=\u201coff\u201d button_border_color__hover_enabled=\u201coff\u201d button_one_border_color__hover_enabled=\u201coff\u201d button_two_border_color__hover_enabled=\u201coff\u201d button_border_radius__hover_enabled=\u201coff\u201d button_one_border_radius__hover_enabled=\u201coff\u201d button_two_border_radius__hover_enabled=\u201coff\u201d button_letter_spacing__hover_enabled=\u201coff\u201d button_one_letter_spacing__hover_enabled=\u201coff\u201d button_two_letter_spacing__hover_enabled=\u201coff\u201d button_bg_color__hover_enabled=\u201coff\u201d button_one_bg_color__hover_enabled=\u201coff\u201d button_two_bg_color__hover_enabled=\u201coff\u201d][\/et_pb_button][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d][et_pb_column type=\u201c4_4\u201d _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d][et_pb_text _builder_version=\u201c4.17.4\u201d _module_preset=\u201cdefault\u201d global_colors_info=\u201d{}\u201d]<\/p>\n<p>#Microsoft\u00adFab\u00adric #Azure #lake\u00adhouse #delta\u00adlake #power\u00adBI #data #ana\u00adlyt\u00adics #AI #onelake #Microsoft<\/p>\n<p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In recent years, Data Lake\u00adhouse archi\u00adtec\u00adture has become the pri\u00adma\u00adry data archi\u00adtec\u00adture for cloud-based data plat\u00adforms. The Medal\u00adlion archi\u00adtec\u00adture (bronze, sil\u00adver, gold) has become the de fac\u00adto stan\u00addard when build\u00ading a&nbsp;Lake\u00adhouse. Until now, Microsoft\u00ad\u2019s solu\u00adtion for cloud-based data plat\u00adforms has been the Azure Synapse Ana\u00adlyt\u00adics PaaS solu\u00adtion or Data\u00adbricks on Azure. Novem\u00adber 2023 Microsoft&nbsp;[\u2026]<\/p>\n","protected":false},"author":20,"featured_media":232862,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_et_pb_use_builder":"on","_et_pb_old_content":"<p>[et_pb_section fb_built=\"1\" _builder_version=\"4.17.4\" _module_preset=\"default\" da_disable_devices=\"off|off|off\" global_colors_info=\"{}\" da_is_popup=\"off\" da_exit_intent=\"off\" da_has_close=\"on\" da_alt_close=\"off\" da_dark_close=\"off\" da_not_modal=\"on\" da_is_singular=\"off\" da_with_loader=\"off\" da_has_shadow=\"on\"][et_pb_row _builder_version=\"4.17.4\" _module_preset=\"default\" global_colors_info=\"{}\"][et_pb_column type=\"4_4\" _builder_version=\"4.17.4\" _module_preset=\"default\" global_colors_info=\"{}\"][et_pb_text _builder_version=\"4.17.4\" _module_preset=\"default\" hover_enabled=\"0\" global_colors_info=\"{}\" sticky_enabled=\"0\"]<\/p><p><strong>Viime vuosina Data Lakehouse -arkkitehtuurista on muodostunut p\u00e4\u00e4s\u00e4\u00e4nt\u00f6inen data-arkkitehtuuri pilvipohjaisiin data-alustoihin. Medallion-arkkitehtuurista (pronssi, hopea, kulta) on tullut de facto Lakehousea rakennettaessa. T\u00e4h\u00e4n asti Microsoftin ratkaisu pilvipohjaisia data-alustoja varten on ollut Azure Synapse Analytics PaaS -ratkaisu tai Databricks on Azure. Viime kuussa Microsoft julkaisi uuden SaaS-pohjaisen analytiikka-alustan nimelt\u00e4 Microsoft Fabric. Jos Fabric ei ole viel\u00e4 sinulle tuttu, <a href=\"https:\/\/learn.microsoft.com\/en-us\/fabric\/get-started\/microsoft-fabric-overview\">tutustu Microsoft Learnin yleiskatsaukseen<\/a>.<\/strong><\/p><p>Alla on muutamia ajatuksiamme Fabricista liittyen Data Lakehouse -toteutuksiin ja miksi organisaatioiden tulisi ottaa Fabric k\u00e4ytt\u00f6\u00f6n.<\/p><p>[\/et_pb_text][et_pb_text _builder_version=\"4.17.4\" _module_preset=\"default\" hover_enabled=\"0\" global_colors_info=\"{}\" sticky_enabled=\"0\"]<\/p><h2><strong>Synapse vs Fabric<\/strong><\/h2><p>Synapse Analyticsin tarkoitus oli tuoda yhden sateenvarjon alle Azuressa saatavilla olevat data-palvelut. Sit\u00e4 se k\u00e4yt\u00e4nn\u00f6ss\u00e4 my\u00f6s teki, mutta konepellin alla ne eiv\u00e4t silti aina toimineet saumattomasti yhteen. N\u00e4ist\u00e4 esimerkkein\u00e4 mm. se, ett\u00e4 Spark- ja SQL-ty\u00f6kuormat eiv\u00e4t keskustele natiivisti kesken\u00e4\u00e4n, yhteydet eri palveluiden v\u00e4lill\u00e4 eiv\u00e4t toimi out of the box ja datasta t\u00e4ytyy tallentaa useampia kopioita, jotta se on optimaalisesti hy\u00f6dynnett\u00e4viss\u00e4 eri ty\u00f6kuormissa. N\u00e4m\u00e4 eiv\u00e4t ole ylitsep\u00e4\u00e4sem\u00e4tt\u00f6mi\u00e4 asioita, mutta aiheuttavat ylim\u00e4\u00e4r\u00e4ist\u00e4 ty\u00f6t\u00e4, joka ei tuota lis\u00e4arvoa loppuk\u00e4ytt\u00e4jille. Toki monet n\u00e4ist\u00e4 asioista pystyt\u00e4\u00e4n hoitamaan automaatiolla, kuten Isletill\u00e4 tekem\u00e4mme template-pohjaiset ratkaisut tekev\u00e4tkin. Molemmissa ratkaisuissa Lakehouse-arkkitehtuurin ytimess\u00e4 on Spark-notebookit, joilla varsinaiset datan k\u00e4sittelyt tehd\u00e4\u00e4n. T\u00e4t\u00e4kin varten olemme kehitt\u00e4neet omat Spark-kirjastot, joiden avulla toteutus on nopeampaa ja laadukkaampaa, ja ne ovat t\u00e4ysin yhteensopivia Fabricin notebookien kanssa.<\/p><h2><strong>Miten Fabric muuttaa kuviota? <\/strong><\/h2><p>Fabric ei muuta Data Lakehousen ja Medallion -arkkitehtuurin perusperiaatteita vaan tarjoaa t\u00e4ysin uudenlaisen alustan n\u00e4iden rakentamista varten. Koska kyseess\u00e4 on SaaS-palvelu, sen pystytt\u00e4minen ja yll\u00e4pit\u00e4minen vaatii v\u00e4hemm\u00e4n ty\u00f6t\u00e4 kuin PaaS-pohjaisen Synapsen. Fabricin yhteinen k\u00e4ytt\u00f6liittym\u00e4 kaikille ty\u00f6kuormille on toki hyv\u00e4 asia ja v\u00e4hent\u00e4\u00e4 tarvittavien ty\u00f6kalujen m\u00e4\u00e4r\u00e4\u00e4 ja siirtymist\u00e4 niiden v\u00e4lill\u00e4. Niin ne ty\u00f6kuormat. Fabricissa on mist\u00e4 valita: Data Factory, Data Engineering + Lakehouse, Data Warehouse, Data Science, Real-time Analytics, Data Activator ja Power BI. N\u00e4ist\u00e4kin voit lukea lis\u00e4\u00e4 em. linkin takaa l\u00f6ytyv\u00e4st\u00e4 Fabricin esittelyst\u00e4. Kaikkia ty\u00f6kuormia ei tietysti tarvitse k\u00e4ytt\u00e4\u00e4 vaan jokaiseen tarpeeseen valitaan sopivin v\u00e4line. Eri ty\u00f6kuormilla pystyt\u00e4\u00e4n toteuttamaan osittain samojakin asioita esim. vaihtoehtoiset toteutukset low-code tai code first -tyyppisesti.<\/p><p><strong>T\u00e4rkein ominaisuus kuitenkin on konepellin alla oleva One Lake tallennustila ja kaikkien Fabric ty\u00f6kuormien k\u00e4ytt\u00e4m\u00e4 Apache Delta Lake tallennusformaatti.<\/strong> One Laken taustalla on Azure Data Lake Storage Gen2, jonka my\u00f6t\u00e4 One Lake tukee kaikkia samoja ominaisuuksia kuin Data Lake Storage. Delta Lake taas on avoin tallennusformaatti, joka tukee ACID-transaktioita sek\u00e4 datan versiointia ja samaista formaattia k\u00e4ytt\u00e4\u00e4 my\u00f6s mm. Databricks. Synapsen Notebookit toki pystyv\u00e4t yht\u00e4 lailla k\u00e4ytt\u00e4m\u00e4\u00e4n Delta Lakea ja Serverless SQL Pool my\u00f6s lukemaan sit\u00e4, mutta <strong>Fabricissa kaikki ty\u00f6kuormat sek\u00e4 lukevat ett\u00e4 kirjoittavat Delta Lakea natiivisti<\/strong>. T\u00e4m\u00e4 tietysti helpottaa eri ty\u00f6kuormien v\u00e4list\u00e4 datan hy\u00f6dynt\u00e4mist\u00e4 ja my\u00f6s eri rooleissa toimivien henkil\u00f6iden mahdollisuutta hy\u00f6dynt\u00e4\u00e4 alustalla olevia datoja eli juuri sit\u00e4, mit\u00e4 modernin data-alustan tulisikin olla.<\/p><p>Yhten\u00e4isen Delta Lake formaatin my\u00f6t\u00e4 tarve kopioida samaa dataa eri formaateissa eri v\u00e4lineit\u00e4 tai k\u00e4ytt\u00f6tarpeita varten v\u00e4henee merkitt\u00e4v\u00e4sti. T\u00e4m\u00e4n lis\u00e4ksi Fabricissa on t\u00e4ysin uusina ominaisuuksina shortcut ja database mirroring, joiden avulla olemassa olevia datoja esim. AWS:n S3:sta, Azuren Storagesta tai Azuren SQL ja Snowflake -tietokannoista ei tarvitse v\u00e4ltt\u00e4m\u00e4tt\u00e4 erikseen siirt\u00e4\u00e4 One Lakeen vaan ne voidaan linkitt\u00e4\u00e4 One Lakeen. Jokainen tapaus tulee tietysti tutkia tarkemmin ja hakea sopivin ratkaisu kyseiseen tarpeeseen.<\/p><p>Uusista ominaisuuksista mainittakoon viel\u00e4 erikseen Power BI:n Direct Lake connector, joka pystyy lukemaan datan One Lakesta reaaliaikaisesti ja eritt\u00e4in suorituskykyisesti eli k\u00e4yt\u00e4nn\u00f6ss\u00e4 yhdist\u00e4\u00e4 parhaat puolet Direct Query ja Import Mode -tyyppisist\u00e4 yhteyksist\u00e4: tietomallin ajantasaisuus ja tehokkuus.<\/p><p>Edell\u00e4 mainittujen lis\u00e4ksi Fabricissa on lukuisia muitakin uusia ominaisuuksia ja tuote kehittyy jatkuvasti. On hyv\u00e4 huomioida, ett\u00e4 vaikka Microsoft julkaisi tuotantokelpoisen (GA) version Fabricista marraskuussa 2023, on sen ominaisuuksissa edelleen puutteita. N\u00e4it\u00e4 kuitenkin paikataan kovaa vauhtia ja uusia ominaisuuksia julkistetaan viikoittain.<\/p><p>[\/et_pb_text][et_pb_text _builder_version=\"4.17.4\" _module_preset=\"default\" hover_enabled=\"0\" global_colors_info=\"{}\" sticky_enabled=\"0\"]<\/p><h2><strong>Milloin on hyv\u00e4 aika ottaa Fabric k\u00e4ytt\u00f6\u00f6n?<\/strong><\/h2><p>Organisaatioiden, jotka vasta aloittavat siirtymisen pilvipohjaiseen data-alustaan, kannattaa ehdottomasti harkita Fabricia ensisijaisena vaihtoehtona. Toisaalta niill\u00e4 organisaatioilla, jotka ovat jo rakentaneet data-alustansa Synapseen tai Databricksiin, ei ole mik\u00e4\u00e4n kiire siirt\u00e4\u00e4 jo tehtyj\u00e4 osia Fabriciin, Synapse pysyy edelleen t\u00e4ysin tuettuna palveluna. Mutta n\u00e4ille organisaatioille voi olla mielenkiintoinen vaihtoehto implementoida Fabric jonkin tietyn osa-alueen k\u00e4ytt\u00f6\u00f6n ja ker\u00e4t\u00e4 siten kokemuksia uudesta alustasta.<\/p><p>Viitteit\u00e4 on, ett\u00e4 migraatioita varten on tulossa Microsoftilta apuv\u00e4lineit\u00e4 jossain vaiheessa. Jos organisaation nykyinen Synapse-pohjainen ratkaisu on Lakehouse Spark-notebookeja k\u00e4ytt\u00e4en, kuten Isletinkin toteutusmalli on, tulee migraatio Fabriciin olemaan melko kevyt operaatio riippumatta siit\u00e4 tekeek\u00f6 sen nyt vai muutaman vuoden kuluttua.<\/p><h2><strong>Kiteytettyn\u00e4, mit\u00e4 hy\u00f6tyj\u00e4 Fabric tuo organisaatiolle?<\/strong><\/h2><p><strong>Saman palvelun alta l\u00f6ytyv\u00e4t nyt kaikki data ja analytiikka -tarpeisiin liittyv\u00e4t asiat<\/strong> datan integroinnista l\u00e4htien sen muokkaamiseen, tallentamiseen ja raportointiin sek\u00e4 lis\u00e4ksi koneoppimis- ja AI-ty\u00f6kalut.<\/p><p>Koska kaikki Fabricin ty\u00f6kalut tunnistavat keskitetyn One Laken ja k\u00e4ytt\u00e4v\u00e4t samaa datan tallennusformaattia, on eri rooleissa ty\u00f6skentelevien henkil\u00f6iden helppo hy\u00f6dynt\u00e4\u00e4 alustaan tallennettua tietoa. Aikaa ja rahaa s\u00e4\u00e4styy, kun henkil\u00f6n ei tarvitse mietti\u00e4, miten saa luettua haluamansa datan.<\/p><p><strong>Samoin ty\u00f6skentely\u00e4 tehostaa Copilot<\/strong>. Se integroidaan osaksi kaikkia Fabricin ty\u00f6kuormia ja sill\u00e4 on k\u00e4yt\u00f6ss\u00e4\u00e4n sama n\u00e4kyvyys alustassa olevaan dataan kuin kehitt\u00e4j\u00e4ll\u00e4, jolloin kehitt\u00e4j\u00e4t voivat pyyt\u00e4\u00e4 Copilotia esim. kirjoittamaan koodia, laskentakaavoja tai analysoimaan dataa.<\/p><p>Fabricin kustannukset perustuvat kapasiteettiyksikk\u00f6ihin, joita kaikki ty\u00f6kuormat kuluttavat. Kun datan m\u00e4\u00e4r\u00e4 kasvaa ja k\u00e4ytt\u00f6tarpeet laajenevat, kapasiteettia ostetaan lis\u00e4\u00e4 tai p\u00e4in vastoin. Power BI -lisenssit tosin ostetaan edelleen erikseen ellei k\u00e4ytet\u00e4 F64-kapasiteettia eli entist\u00e4 Power BI Premiumia.<\/p><h2><strong>Islet ja Fabric<\/strong><\/h2><p>Me Isletill\u00e4 olemme jo pitk\u00e4n aikaa toteuttaneet Data Lakehouse arkkitehtuureita perinteisten tietovarastojen sijaan, kuten esim. <a href=\"https:\/\/isletgroup.fi\/2023\/11\/08\/wihuri-uudistaa-tietojenhallintaa-ja-analytiikkaa-isletin-avulla\/\"><strong>Wihurilla<\/strong><\/a>. Olemme kehitt\u00e4neet geneerisi\u00e4, toistettavia malleja ja kirjastoja Medallion-arkkitehtuurin tehokkaaseen toteutukseen ja k\u00e4yt\u00e4mme Delta Lakea datan tallennusformaattina. N\u00e4m\u00e4 huomioiden, siirtyminen Fabriciin ei muuta suuresti tapaamme toteuttaa Lakehouse, mutta tuo paljon uusia mahdollisuuksia ja ominaisuuksia data-alustan rakentamista ja datan hy\u00f6dynt\u00e4mist\u00e4 varten.<\/p><p>[\/et_pb_text][et_pb_text _builder_version=\"4.17.4\" _module_preset=\"default\" hover_enabled=\"0\" global_colors_info=\"{}\" sticky_enabled=\"0\"]<\/p><p>- - - - -<\/p><p><em>Blogin <span class=\"ui-provider a b c d e f g h i j k l m n o p q r s t u v w x y z ab ac ae af ag ah ai aj ak\" dir=\"ltr\">kirjoittaja Mika Kuivanen on Isletin data-arkkitehti, jolla on yli 15 vuoden kokemus tietokannoista, datasta&analytiikasta ja konsultoinnista. \u00a0<\/span><\/em> \u00a0<\/p><p>Lis\u00e4tietoja:<\/p><p>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row column_structure=\"1_4,1_4,1_2\" _builder_version=\"4.16\" background_size=\"initial\" background_position=\"top_left\" background_repeat=\"repeat\" global_colors_info=\"{}\"][et_pb_column type=\"1_4\" _builder_version=\"4.16\" custom_padding=\"|||\" global_colors_info=\"{}\" custom_padding__hover=\"|||\"][\/et_pb_column][et_pb_column type=\"1_4\" _builder_version=\"4.16\" custom_padding=\"|||\" global_colors_info=\"{}\" custom_padding__hover=\"|||\"][et_pb_image src=\"https:\/\/isletgroup.fi\/wp-content\/uploads\/2021\/02\/Janne_Anttila_Beanie_Round.png\" title_text=\"Janne_Anttila_Beanie_Round\" url_new_window=\"on\" align=\"center\" align_tablet=\"center\" align_phone=\"\" align_last_edited=\"on|desktop\" _builder_version=\"4.16\" width=\"85%\" global_colors_info=\"{}\"][\/et_pb_image][et_pb_text _builder_version=\"4.17.4\" custom_margin=\"|-10px||-30px|false|false\" global_colors_info=\"{}\"]<\/p><p style=\"text-align: center;\">Janne Anttila<\/p><p style=\"text-align: center;\">CBO - Data and Analytics, Isletter<\/p><p style=\"text-align: center;\"><a href=\"mailto:janne.anttila@isletgroup.fi\">janne.anttila@isletgroup.fi<\/a><\/p><p style=\"text-align: center;\">+358 45 672 8569<\/p><p>[\/et_pb_text][\/et_pb_column][et_pb_column type=\"1_2\" _builder_version=\"4.16\" custom_padding=\"|||\" global_colors_info=\"{}\" custom_padding__hover=\"|||\"][et_pb_text admin_label=\"more information text (fill it if you want to\" _builder_version=\"4.16\" custom_margin=\"||40px|||\" global_colors_info=\"{}\"][\/et_pb_text][et_pb_button button_url=\"mailto:janne.anttila@isletgroup.fi\" url_new_window=\"on\" button_text=\"Ota yhteytt\u00e4\" button_alignment=\"center\" admin_label=\"email\/get in touch button\" module_class=\"CTA_box_bedrock\" _builder_version=\"4.16\" custom_margin=\"140px||||false|false\" global_colors_info=\"{}\" button_text_size__hover_enabled=\"off\" button_one_text_size__hover_enabled=\"off\" button_two_text_size__hover_enabled=\"off\" button_text_color__hover_enabled=\"off\" button_one_text_color__hover_enabled=\"off\" button_two_text_color__hover_enabled=\"off\" button_border_width__hover_enabled=\"off\" button_one_border_width__hover_enabled=\"off\" button_two_border_width__hover_enabled=\"off\" button_border_color__hover_enabled=\"off\" button_one_border_color__hover_enabled=\"off\" button_two_border_color__hover_enabled=\"off\" button_border_radius__hover_enabled=\"off\" button_one_border_radius__hover_enabled=\"off\" button_two_border_radius__hover_enabled=\"off\" button_letter_spacing__hover_enabled=\"off\" button_one_letter_spacing__hover_enabled=\"off\" button_two_letter_spacing__hover_enabled=\"off\" button_bg_color__hover_enabled=\"off\" button_one_bg_color__hover_enabled=\"off\" button_two_bg_color__hover_enabled=\"off\"][\/et_pb_button][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=\"4.17.4\" _module_preset=\"default\" global_colors_info=\"{}\"][et_pb_column type=\"4_4\" _builder_version=\"4.17.4\" _module_preset=\"default\" global_colors_info=\"{}\"][et_pb_text _builder_version=\"4.17.4\" _module_preset=\"default\" global_colors_info=\"{}\"]\u00a0<\/p><p>#MicrosoftFabric #Azure #lakehouse #deltalake #powerBI #data #analytiikka #AI #onelake #Microsoft<br \/>[\/et_pb_text][\/et_pb_column][\/et_pb_row][\/et_pb_section]<\/p>","_et_gb_content_width":"","wp_typography_post_enhancements_disabled":false,"footnotes":""},"categories":[453],"tags":[],"class_list":["post-232908","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cover-story"],"acf":[],"_links":{"self":[{"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/posts\/232908","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/users\/20"}],"replies":[{"embeddable":true,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/comments?post=232908"}],"version-history":[{"count":7,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/posts\/232908\/revisions"}],"predecessor-version":[{"id":233570,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/posts\/232908\/revisions\/233570"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/media\/232862"}],"wp:attachment":[{"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/media?parent=232908"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/categories?post=232908"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/isletgroup.fi\/en\/wp-json\/wp\/v2\/tags?post=232908"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}